data-quality-checks

What problem does it solve? Bad data silently corrupts pipelines, dashboards, and downstream decisions. This Skill automates data quality validation so schema drift, nulls, duplicates, stale data, and anomalies are caught on every pipeline run instead of after users notice. ## Core Features & Use Cases - Great Expectations Validation: Build expectation suites covering schema, types, nullability, uniqueness, and value ranges, then run them as checkpoints on every ETL execution. - Anomaly & Freshness Detection: Detect statistical outliers with Z-scores, flag unusual daily volumes against rolling baselines, and enforce data freshness SLAs. - Quarantine & Lineage: Separate invalid records into quarantine storage for debugging, track transformations from source to destination, and expose quality metrics via Prometheus. - Use Case: An hourly Airflow DAG loads orders data, runs six automated checks (schema, completeness, uniqueness, ranges, freshness, anomalies), records a quality score, and fails the pipeline if critical checks fail. ## Quick Start Ask the AI to design automated data quality checks with Great Expectations for your orders dataset, including schema validation, freshness SLAs, and quarantine handling.