skills.rest
aliyun
aliyunOFFICIAL
Explore all skills by this author →
WebsiteGitHub

data-profiling

Generates descriptive analytics reports for CSV or Parquet datasets.

580|66|Updated Apr 21, 2025
Search Tags:#data quality#report generation#data profiling#schema inference#dataset exploration#missing value#CSV Parquet
One-click install
npx skills add https://github.com/aliyun/qwen-dianjin --skill data-profiling
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-profiling
Source: https://github.com/aliyun/qwen-dianjin/tree/main/DianJin-SKILLS/financial-engineering-expert/data-profiling
Command: npx skills add https://github.com/aliyun/qwen-dianjin --skill data-profiling

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pandas, numpy, and includes scripts (resource) components.

What problem does it solve?

When you upload a dataset and need to quickly understand its shape and data quality, this Skill generates a clear data profiling report (rows, columns, field types, missing rates, distributions) without running any modeling metrics.

Core Features & Use Cases

  • Dataset structure overview: Produces a fast report including file info, row/column counts, and inferred field-type distribution.
  • Field-level description: Lists each column’s inferred type, missing rate, unique value count, and a few example values.
  • Quality & preview insights: Highlights issues such as duplicate rows, empty/constant columns, and high-missing-rate fields, plus a small sample preview.
  • Use case: After you receive a financial dataset file (CSV/Parquet), run this Skill to decide what to do next (e.g., whether you need deeper feature analysis or modeling).

Quick Start

Ask the agent to run data-profiling on your dataset file path to generate a profiling markdown report and a structured result manifest for downstream steps.

Frequently Asked Questions about data-profiling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate a data profiling report for a CSV or Parquet dataset?▼

Data profiling reports describe dataset structure, field types, missing rates, and distributions without running modeling metrics. Run the profiler script on your CSV or Parquet file path to output a markdown report and structured JSON manifest.

What is the best way to explore a dataset structure before modeling?▼

Dataset exploration via data profiling produces a fast report including row and column counts, inferred field types, missing rates, and example values. It focuses on descriptive insight rather than target-variable-dependent modeling metrics.

Can I detect missing values and duplicate rows in a dataset using pandas?▼

Yes, you can detect missing values and duplicate rows by running the data profiling script, which uses pandas and numpy to highlight quality issues like empty or constant columns, high missing rates, and duplicate rows.

Does data profiling work with both CSV and Parquet files?▼

Data profiling works with both CSV and Parquet files, scanning the dataset to produce a markdown report and a structured result.json manifest for downstream steps without requiring any modeling metrics.

What are the limitations of using data profiling for dataset exploration?▼

Data profiling is limited to descriptive insights like structure, field types, and missing rates, and explicitly avoids target-variable-dependent metrics, making it unsuitable if you need deeper feature analysis or predictive modeling.

Related Skills

Explore More Data & Analytics Skills→
swarm-ai-researchswarm-ai-research

Programmatic EDA

Profile local CSV, Parquet, and Excel datasets with pandas and numpy, outputting Markdown reports.

Official
Intermediate
xchang1121xchang1121
6

performance-summary

Aggregate performance metrics from verify_<op_name>/result.json files across operators.

Community
Intermediate
sencersoylusencersoylu

descriptive-statistics

Compute descriptive statistics and Table 1 summaries from datasets using Python.

Community
Intermediate
williamzujkowskiwilliamzujkowski
17

data-quality

Validate dataset completeness, accuracy, and consistency with Great Expectations.

Community
Advanced