bigconfig-generator

Data quality monitoring configuration automates the creation of metadata.yaml and bigconfig.yml files that define freshness, volume, nulls, and uniqueness checks for BigQuery tables. This Skill generates those configurations using templates aligned with table criticality, reducing manual setup and ensuring consistency across Mozilla's data pipelines.

Yes, this Skill generates both standard checks and optional custom SQL rules through bigeye_custom_rules.sql. You provide your table scenario and monitoring requirements, and it produces the full configuration bundle ready for deployment in the bigquery-etl repository.

The Skill enforces a safe deployment workflow: generate configurations, update metadata.yaml and bigconfig.yml templates, coordinate with metadata-manager and bqetl-core for validation, then commit through automated deployment rather than manual patching. This prevents configuration drift and deployment errors.

Yes, this Skill integrates with metadata-manager to align monitoring configuration with schema and metadata generation. Coordinating these ensures your BigQuery table definitions, metadata, and monitoring checks stay synchronized throughout your data pipeline.

Generated configurations include standard data quality checks for freshness, volume, null values, and uniqueness. You can extend these with optional custom SQL rules tailored to your specific table requirements and data quality thresholds.