schema-enforcer

What problem does it solve?

This Skill addresses the challenge of maintaining data integrity and consistency in data-intensive pipelines by providing tools to automatically generate and validate data schemas.

Core Features & Use Cases

• Pydantic Model Generation: Automatically create Pydantic models from Parquet or YAML files, ensuring type safety and structure.
• DataFrame Validation: At runtime, validate Polars DataFrames against defined Pydantic models to catch schema violations early.
• Use Case: Ensure that data ingested into a data warehouse conforms to a predefined structure, preventing downstream errors and data quality issues.

Quick Start

Generate a Pydantic model named 'CustomerSchema' from the Parquet file located at '/data/customer.parquet' with nullable fields.