data-quality-frameworks

To enforce data quality, you integrate Great Expectations validation, dbt tests, and data contracts to catch data issues early across production and CI/CD workflows. This framework provides patterns for validating source data and ETL outputs.

Data contracts are deterministic checks used for schema validation to ensure reliable pipelines. You need them when validating datasets and monitoring quality metrics to enforce agreements between data producers and consumers.

Yes, you can automate data validation orchestrations across CI/CD workflows. The framework supports automating validations to monitor quality metrics and catch data issues early in production environments.

The best way to validate ETL outputs is by applying unit, integration, and contract-based validations. This approach monitors quality metrics across pipelines using Great Expectations and dbt tests.

You do not need dbt exclusively, as Great Expectations handles deterministic checks independently. However, integrating dbt tests with data contracts provides comprehensive schema validation and automated validation orchestrations.

Data contracts provide schema validation and enforce deterministic checks beyond standard dbt tests. Combining them with Great Expectations ensures comprehensive data quality across pipelines, validating both source data and ETL outputs.