What problem does it solve? Designing a production data pipeline from scratch requires coordinating extractors, transformers, loaders, orchestration, quality checks, and monitoring, which is slow and error-prone when done manually. ## Core Features & Use Cases - End-to-End Pipeline Generation: Produces concrete Python extractor, transformer, and loader classes plus SQL files with no placeholder code. - Orchestration & Monitoring Setup: Generates a wired Airflow DAG, pipeline config YAML, alert rules, and metrics collection utilities. - Data Quality Framework: Creates composable checks (not-null, unique, freshness, referential integrity, schema) with severity levels and thresholds. - Use Case: You need to move daily orders from Postgres into BigQuery with deduplication and freshness SLAs. Describe the sources and destination, approve the design, and receive the full project layout, DAG, tests, and a data-pipeline-spec.md runbook. ## Quick Start Ask the assistant to design and build an ETL pipeline from your Postgres orders database into BigQuery with daily scheduling and data quality checks.
