data-validation

Enforce data quality boundaries with schema validation, type checks, and business rules.

Updated Feb 20, 2026

Search Tags:#pydantic #schema #etl #data-quality #great-expectations #data-validation #data-contracts

Ready to run

npx skills add https://github.com/saajunaid/junai --skill data-validation-saajunaid

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: data-validation
Source: https://github.com/saajunaid/junai/tree/main/.github/skills/data/data-validation
Command: npx skills add https://github.com/saajunaid/junai --skill data-validation-saajunaid

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Data quality across ingestion, transformation, and output is enforceable with consistent validation patterns at every boundary to prevent bad data from propagating.

Core Features & Use Cases

Pydantic-based schema validation for Python data models and API inputs.
Great Expectations-inspired checks and SQL-level constraints to codify data contracts.
End-to-end validation strategy across ETL pipelines, data processing jobs, and data outputs.

Quick Start

Define a simple, repeatable data validation plan and run it against a sample dataset to see immediate improvements.

Frequently Asked Questions about data-validation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I enforce data validation across ETL pipelines?▼

Data validation across ETL pipelines is enforced by applying schema validation, type checks, and business rules at ingestion, transformation, and output boundaries. This prevents bad data from propagating by using Pydantic models and SQL constraints.

What is the best way to define data contracts and catch bad records?▼

Data contracts are defined by codifying schema expectations and business rules using Pydantic models and Great Expectations patterns. When data fails these checks, dead-letter records are surfaced and alerts are triggered to catch bad records.

How do I validate API inputs using Pydantic schema checks?▼

API inputs are validated using Pydantic-based schema validation for Python data models. This approach applies strict type checks and structural constraints to ensure incoming API payloads meet defined data quality boundaries.

Can I use Great Expectations patterns for batch validation in data processing jobs?▼

Great Expectations-inspired checks can be applied for batch validation across data processing jobs. The validation strategy performs checks on datasets, surfacing failures and dead-letter records when data does not meet the defined constraints.

How do SQL constraints prevent bad data from propagating in data pipelines?▼

SQL constraints prevent bad data propagation by enforcing strict schema and type rules directly within the pipeline transformation and output stages. Combining them with Python models ensures end-to-end data quality across all boundaries.

Why do I need data validation contracts if I already have ETL transformations?▼

Data validation contracts are needed because transformations alone do not guarantee data quality. Explicitly defining and enforcing contracts at every boundary detects and prevents bad data, ensuring consistent schema validation and type checks across the pipeline.