skills.rest
wshobson
wshobsonCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

data-quality-frameworks

Implement data quality validation frameworks using Great Expectations, dbt tests, and data contracts.

38.6k|4.1k|Updated Jul 24, 2025
Search Tags:#validation#data quality#dbt#pipeline#data contracts#great expectations
One-click install
npx skills add https://github.com/wshobson/agents --skill data-quality-frameworks-wshobson
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-quality-frameworks
Source: https://github.com/wshobson/agents/tree/main/plugins/data-engineering/skills/data-quality-frameworks
Command: npx skills add https://github.com/wshobson/agents --skill data-quality-frameworks-wshobson

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the critical need for reliable data by implementing comprehensive validation strategies, preventing data corruption and ensuring data trustworthiness in pipelines.

Core Features & Use Cases

  • Data Quality Validation: Implement checks using Great Expectations, dbt tests, and data contracts.
  • Pipeline Integration: Seamlessly integrate data quality checks into your data pipelines.
  • Use Case: When building a new data pipeline for customer orders, use this Skill to set up Great Expectations to validate that order_id is unique and not null, and dbt tests to ensure referential integrity with the customers table.

Quick Start

Use the data-quality-frameworks skill to set up Great Expectations validation for the 'orders' table using the 'orders_suite' expectation suite.

Frequently Asked Questions about data-quality-frameworks

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I implement data quality validation in my data pipeline?▼

You can implement data quality validation using Great Expectations, dbt tests, and data contracts to ensure completeness, uniqueness, validity, accuracy, consistency, and timeliness through automated checks.

What is the best way to set up Great Expectations for a dbt pipeline?▼

The best way to set up data quality validation is to integrate Great Expectations and dbt tests directly into your pipeline, establishing expectation suites for tables like 'orders' to validate uniqueness and referential integrity.

How do data contracts prevent data corruption in pipelines?▼

Data contracts prevent data corruption by establishing strict validation rules and expectations between data producers and consumers, ensuring trustworthiness and pipeline integrity before data reaches downstream systems.

Can I use dbt tests to check referential integrity with Great Expectations?▼

Yes, you can use dbt tests to ensure referential integrity between tables like 'orders' and 'customers', while simultaneously using Great Expectations to validate that critical fields such as order_id are unique and not null.

Does this approach support automated checks for data completeness and accuracy?▼

Yes, the framework supports automated checks specifically designed to satisfy requirements for ensuring data completeness, uniqueness, validity, accuracy, consistency, and timeliness within your data pipelines.

Related Skills

Explore More Software Engineering Skills→
allthingslinuxallthingslinux
4

designing-zod-schemas

Design Zod schemas for validation and TypeScript type inference.

Official
Intermediate
dobeutechdobeutech
1

listener-creator

Generate TypeScript email listeners that monitor events and execute custom actions.

Community
Advanced
emilneuraz-aiemilneuraz-ai

data-quality-frameworks

Enforce data quality across pipelines with Great Expectations, dbt tests, and data contracts.

Community
Advanced
PriyanshKuniyalPriyanshKuniyal

data-quality-frameworks

Automate data quality validation with Great Expectations and dbt tests.

Community
Advanced