skills.rest
seb1n
seb1nCOMMUNITY
Explore all skills by this author →
GitHub

data-cleaning

Clean datasets with pandas, pyjanitor, and great_expectations.

147|32|Updated Feb 11, 2026
Search Tags:#validation#data quality#pandas#data cleaning#data preprocessing#outlier detection
One-click install
npx skills add https://github.com/seb1n/awesome-ai-agent-skills --skill data-cleaning-seb1n
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-cleaning
Source: https://github.com/seb1n/awesome-ai-agent-skills/tree/main/data-and-analytics/data-cleaning
Command: npx skills add https://github.com/seb1n/awesome-ai-agent-skills --skill data-cleaning-seb1n

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pandas, pyjanitor, great_expectations, fuzzywuzzy, numpy, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill tackles the pervasive issue of messy, inconsistent, and incomplete data, transforming raw datasets into a reliable format ready for analysis and machine learning.

Core Features & Use Cases

  • Handles Missing Values: Imputes or flags missing data based on column type and missingness patterns.
  • Deduplication: Identifies and removes exact and near-duplicate records.
  • Type Coercion & Standardization: Corrects data types and standardizes formats (dates, strings, numbers).
  • Outlier Detection & Treatment: Identifies and handles outliers using statistical methods.
  • Schema Validation: Enforces data quality rules to ensure consistency.
  • Use Case: Clean a customer database with missing emails, duplicate entries, and inconsistent region names before running a marketing campaign.

Quick Start

Use the data-cleaning skill to clean the attached file 'customer_data.csv' and save the output to 'cleaned_customer_data.csv'.

Frequently Asked Questions about data-cleaning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I clean messy datasets with missing values and duplicate records for machine learning?▼

To clean messy datasets for machine learning, you can handle missing values, remove duplicates, and correct data types. This process transforms raw, inconsistent data into a reliable format ready for analysis and modeling.

Can I enforce schema validation and data quality rules on a pandas DataFrame?▼

Yes, you can enforce schema validation on a pandas DataFrame using great_expectations. It enforces data quality rules to ensure consistency and validates your dataset against expected formats and constraints.

What's the best way to detect and treat outliers in a CSV file before data analysis?▼

The best way to detect and treat outliers in a CSV file is by using statistical methods to identify anomalies. The data cleaning process resolves outliers and standardizes formats to prepare the dataset for analysis.

Does this data preprocessing approach work with fuzzy matching for inconsistent string entries?▼

Yes, data preprocessing works with fuzzy matching for inconsistent strings using fuzzywuzzy. It identifies and removes near-duplicate records, correcting inconsistent region names or customer entries before analysis.

How do I correct data types and standardize formats across multiple columns in a dataset?▼

To correct data types and standardize formats, the data cleaning process uses pandas and pyjanitor to coerce types and standardize dates, strings, and numbers, ensuring your dataset maintains consistent formatting.

Related Skills

Explore More Data & Analytics Skills→
JayRHaJayRHa
3

pandas-data-cleaning

Clean messy tabular datasets with pandas using auditable transformation steps.

Community
Advanced
saeed-vayghansaeed-vayghan
30

data-scientist

Analyze datasets and build predictive models for business insights.

Community
Advanced
githubgithub
37.6k

powerbi-modeling

Guide Power BI semantic model design, DAX measures, and performance optimization.

Official
Advanced
timequitytimequity
6

pipeline-design

Design ETL/ELT pipeline architectures for batch and streaming data integration.

Official
Advanced