skills.rest
cyyeh
cyyehCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

data-exploration

Profile datasets to summarize structure, quality, distributions, and anomalies.

1|Updated Feb 19, 2026
Search Tags:#data quality#data profiling#outlier detection#schema discovery#exploratory analysis#column statistics#correlation analysis
One-click install
npx skills add https://github.com/cyyeh/duckdb-data-agent --skill data-exploration-cyyeh
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-exploration
Source: https://github.com/cyyeh/duckdb-data-agent/tree/main/plugins/data/skills/data-exploration
Command: npx skills add https://github.com/cyyeh/duckdb-data-agent --skill data-exploration-cyyeh

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Provides a systematic process to quickly understand a dataset's shape, quality, and patterns so analysts can decide whether data is usable and which transformations or investigations are needed before analysis.

Core Features & Use Cases

  • Structural profiling: Row and column counts, grain, primary key candidates, and schema overview to establish how the table is organized.
  • Column-level statistics: Null rates, distinct counts, top/bottom values, lengths for text, percentiles and distribution summaries for numeric fields, and date range checks.
  • Quality and pattern discovery: Completeness scoring, consistency checks, placeholder detection, outlier spotting, correlation matrices, and foreign-key/hierarchy candidates for downstream joins.
  • Use Case: When onboarding a new CSV export or data lake table, run the profile to discover surprising nulls, mixed types, skewed distributions, and candidate keys before building dashboards or joining tables.

Quick Start

Open the dataset and run a profile to get column-level stats, null rates, top values, and suggested data types.

Frequently Asked Questions about data-exploration

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I profile a CSV dataset to check data quality and find anomalies?▼

To profile a CSV dataset for data quality, run a structural scan to generate column-level statistics, null rates, distinct counts, and outlier detection. This process evaluates completeness and spots anomalies like mixed types or skewed distributions before downstream analysis.

What is exploratory data analysis and when do I need schema discovery?▼

Exploratory data analysis is the process of summarizing a dataset's structure, distributions, and correlations to determine its usability. You need schema discovery when onboarding new tables, exports, or files to identify primary key candidates and understand table organization before querying.

Can I run column statistics and correlation analysis on Parquet and Excel files?▼

Yes, you can run column statistics and correlation analysis on Parquet and Excel files, along with CSV, JSON, and database tables. The profiling generates distribution summaries, percentile checks, and correlation matrices suitable for preparing data for SQL queries.

What's the best way to detect outliers and null rates before building dashboards?▼

The best way to detect outliers and null rates is to run a comprehensive data profile that calculates completeness scores, identifies placeholders, and generates distribution summaries. This reveals surprising nulls and skewed distributions before you build dashboards or join tables.

Does data profiling automatically suggest foreign-key candidates for joining tables?▼

Yes, data profiling automatically suggests foreign-key and hierarchy candidates during the quality and pattern discovery phase. By analyzing column statistics and distinct counts, it identifies potential join keys to guide downstream SQL analysis and table integration.

Why does my dataset profile show mixed types and how do I fix placeholder values?▼

A dataset profile shows mixed types and placeholder values when consistency checks detect non-standard entries masquerading as valid data. Use the generated completeness scores and placeholder detection results to apply necessary transformations and clean the dataset.

Related Skills

Explore More Data & Analytics Skills→
bettercallfanbettercallfan

dataset-onboarding-graph-preprocess

Convert raw phone network data files into structured graph tables.

Community
Advanced
tmorrowdevtmorrowdev
1

data-exploration

Profile tabular datasets to surface structural metadata and column-level statistics.

Community
Intermediate
markus41markus41
12

pandas-cleaning

Clean and standardize messy tabular data into normalized DataFrames and formatted .xlsx files.

Community
Advanced
hung-phanhung-phan
1

eda

Perform exploratory data analysis with statistical summaries, correlation, and outlier detection.

Community
Advanced