great-expectations

What problem does it solve?

Great Expectations (GX) helps you prevent broken or low-quality data from silently flowing through data pipelines by providing repeatable, testable validation checks with actionable results.

Core Features & Use Cases

• Build data quality rules: Create and maintain Expectation Suites for tables, DataFrames, or files (e.g., null checks, uniqueness, ranges, regex/pattern checks, type checks).
• Run validations reliably: Define ValidationDefinitions and execute them via Checkpoints that produce structured pass/fail outcomes.
• Produce quality reports: Generate Data Docs and integrate validations into orchestration and analytics workflows (e.g., Airflow, dbt, Spark, SQL backends, CI/CD), including support for custom expectations and severity levels (warning/critical).

Quick Start

Ask the agent to set up a file-based GX DataContext, define a dataset batch (from Pandas/Spark/SQL/file), create an Expectation Suite with null/range/type checks, and run a Checkpoint to generate Data Docs for the latest batch.