skills.rest
Datus-ai
Datus-aiOFFICIAL
Explore all skills by this author →
WebsiteGitHub

data-profiler

Profile data quality metrics and value distributions for tabular datasets.

1.5k|222|Updated Jul 4, 2025
Search Tags:#data quality#data validation#anomaly detection#data profiling#tabular data#distribution analysis
One-click install
npx skills add https://github.com/Datus-ai/Datus-agent --skill data-profiler-datus-ai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-profiler
Source: https://github.com/Datus-ai/Datus-agent/tree/main/tests/data/skills/data-profiler
Command: npx skills add https://github.com/Datus-ai/Datus-agent --skill data-profiler-datus-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Manually validating data quality, identifying null values, and understanding column value distributions for large tabular datasets is time-consuming and prone to human error, creating bottlenecks for data engineering and analytics teams.

Core Features & Use Cases

  • Automated Quality Checks: Calculate null percentages and identify unique columns to flag data completeness and uniqueness issues without manual querying.
  • Distribution Insights: Generate row counts and high-level distribution metrics to quickly understand dataset structure and size.
  • Use Case: A data engineer onboarding a new raw customer data table can use this skill to rapidly validate that the dataset has acceptable null rates and expected unique identifier columns before building downstream transformation pipelines.

Quick Start

Use the data-profiler skill to generate a quality and distribution profile for the raw sales_orders table.

Frequently Asked Questions about data-profiler

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate data validation and null percentage checks for tabular datasets?▼

Automated data validation calculates null percentages and identifies unique columns to flag completeness issues in tabular datasets. This replaces manual querying by generating standardized profile outputs like row counts and null rates for raw tables.

What is data profiling and how does it help monitor data drift in pipelines?▼

Data profiling is the process of analyzing value distributions and anomalies in structured data sources. It supports data drift monitoring by generating baseline distribution metrics and row counts that can be compared against new pipeline inputs over time.

Can I use data profiling to identify unique column identifiers before building transformation pipelines?▼

Yes, data profiling identifies unique columns within raw tabular datasets. Data engineers use these standardized profile outputs to confirm expected unique identifier columns and acceptable null rates before constructing downstream transformation pipelines.

What's the best way to check data quality metrics when onboarding a new raw table?▼

The best way to check data quality metrics during raw table onboarding is to run an automated profiling process. It instantly evaluates row counts, null percentages, and value distributions to eliminate manual data validation work.

Does automated data profiling work for anomaly detection in structured data sources?▼

Automated data profiling applies to anomaly detection by analyzing value distributions within structured data sources. It profiles standardized metrics across tabular datasets to surface unexpected distribution shifts and completeness issues.

Related Skills

Explore More Data & Analytics Skills→
brainbrewlabsbrainbrewlabs
7

data-cleaning

Profile and clean tabular datasets, then write a cleaned file with a change report.

Official
Intermediate
SteadfastAsArtSteadfastAsArt
46

lasio

Parse and normalize LAS well-log files into DataFrames or CSV.

Community
Intermediate
huggingfacehuggingface
10.9k

hugging-face-trackio

Log, query, and visualize ML training metrics via Python API and CLI.

Official
Intermediate
hung-phanhung-phan
1

eda

Perform exploratory data analysis with statistical summaries, correlation, and outlier detection.

Community
Advanced