skills.rest
GeneralReasoning
GeneralReasoningOFFICIAL
Explore all skills by this author →
Website𝕏@GenReasoningGitHub

data_cleaning

Automate tabular dataset cleaning with deduplication, imputation, text normalization, and outlier handling.

4|Updated Feb 22, 2026
Search Tags:#pipeline#pandas#deduplication#data-cleaning#imputation#outliers#text-processing
One-click install
npx skills add https://github.com/GeneralReasoning/env-skillsbench --skill data-cleaning-generalreasoning
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data_cleaning
Source: https://github.com/GeneralReasoning/env-skillsbench/tree/main/trend-anomaly-causal-inference/environment/skills/data_cleaning
Command: npx skills add https://github.com/GeneralReasoning/env-skillsbench --skill data-cleaning-generalreasoning

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pandas, numpy, scikit-learn, and includes scripts (resource) components.

What problem does it solve?

Messy tabular datasets often suffer from duplicates, missing values, inconsistent text, and outliers, which degrade analytics, model performance, and reporting accuracy.

Core Features & Use Cases

  • Deduplication: remove duplicates to ensure data integrity across records.
  • Missing value handling: drop or impute missing data with median or mode, enabling downstream analysis.
  • Text processing: normalize and extract useful text signals from columns.
  • Outlier handling: cap or remove extreme values to stabilize statistics.
  • Pipeline orchestration: build reusable CleaningPipeline to apply multiple strategies in sequence.

Quick Start

Create a DataCleaningPipeline, chain steps such as remove_duplicates, impute_median, process_text, and cap_outliers_iqr, then execute it on your DataFrame to obtain a clean dataset.

Frequently Asked Questions about data_cleaning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate pandas data cleaning for messy tabular datasets?▼

Build a DataCleaningPipeline to sequentially apply remove_duplicates, impute_median, process_text, and cap_outliers_iqr on your DataFrame. This pipeline framework automates messy tabular dataset cleaning end-to-end.

What is the best way to handle missing values and outliers in a data preparation pipeline?▼

Use impute_knn, impute_median, or drop_missing to handle missing values, and apply cap_outliers_iqr or remove_outliers_zscore to manage outliers. These modular CleaningStrategies stabilize statistics within a data preparation pipeline.

Can I use scikit-learn and numpy for data imputation in pandas dataframes?▼

Yes, the framework uses scikit-learn and numpy to support impute_knn and impute_median functions. These strategies directly process pandas DataFrames to fill missing values for varied data schemas.

How does text normalization work for inconsistent text columns in tabular data?▼

Text normalization works via the process_text function, which cleans and extracts useful text signals from raw string columns. This ensures consistent formatting across records before downstream analytics or ML modeling.

Does this data cleaning pipeline support deduplication across varied schemas?▼

Yes, the pipeline includes the remove_duplicates function to ensure data integrity across records. It applies effectively to varied schemas and incomplete data across different domains.

When should I drop missing data instead of using imputation in my pipeline?▼

Use the drop_missing function when incomplete records cannot support analysis. Choose drop_missing over impute_knn or impute_median when missing values are too extensive to fill reliably for downstream analytics.

Related Skills

Explore More Data & Analytics Skills→
JayRHaJayRHa
3

pandas-data-cleaning

Clean messy tabular datasets with pandas using auditable transformation steps.

Community
Advanced
borgheiborghei
467

data-scientist

Automate data science workflows for forecasting, segmentation, and optimization tasks.

Community
Advanced
dkyazzentwatwadkyazzentwatwa
86

feature-engineering-kit

Automate feature generation for machine learning pipelines with scikit-learn, pandas, and numpy.

Community
Intermediate
oabdelmaksoudoabdelmaksoud
3

regex-vs-llm-structured-text

Parses structured text using regex with LLM fallback for low-confidence cases.

Community
Intermediate