airflow-starrocks-data-quality
CommunityGate StarRocks loads with trusted DQ checks
Data & Analytics#data quality#airflow#starrocks#freshness checks#dq gates#partition quarantine#z-score anomaly detection
Authorivanshamaev
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill helps prevent broken or incomplete StarRocks data from reaching downstream pipelines by adding automated data quality gates directly after loads.
Core Features & Use Cases
- Post-load DQ validation: Verifies row counts, data freshness, null-rate thresholds, and duplicate-key violations after Broker Load or Stream Load.
- Anomaly and integrity checks: Detects volume anomalies using a z-score against recent history and validates referential integrity between fact and dimension tables.
- Quarantine bad partitions: Moves failing partition data to a quarantine table and drops the bad partition to protect downstream analytical models.
Quick Start
Use the Skill by running its DQ Gate DAG in Airflow to validate an orders partition (row count, freshness, null rates, duplicates, volume anomaly, and FK orphan rate) before allowing downstream tasks to execute.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: airflow-starrocks-data-quality Download link: https://github.com/ivanshamaev/de-agent-skills/archive/main.zip#airflow-starrocks-data-quality Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.