skills.rest
ivanshamaev
ivanshamaevCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

starrocks-data-quality-guardian

Run SQL-based freshness, duplicate, null, volume drift, integrity, and completeness checks on StarRocks tables.

14|1|Updated May 5, 2026
Search Tags:#data quality#duplicate detection#starrocks#SQL validation#freshness checks#null anomaly#dq monitoring
One-click install
npx skills add https://github.com/ivanshamaev/de-agent-skills --skill starrocks-data-quality-guardian
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: starrocks-data-quality-guardian
Source: https://github.com/ivanshamaev/de-agent-skills/tree/main/group_skills/starrocks_group_skills/starrocks_data_quality_guardian
Command: npx skills add https://github.com/ivanshamaev/de-agent-skills --skill starrocks-data-quality-guardian

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pymysql, and includes references (resource) components.

What problem does it solve?

StarRocks data quality problems often surface late—after downstream jobs fail or dashboards lie—so freshness, duplicates, null anomalies, volume drift, integrity, and storage-health issues need consistent, lightweight detection.

Core Features & Use Cases

  • Freshness monitoring: Detect stale data via MAX(updated_at) against an SLA, including per-partition freshness checks.
  • Anomaly detection: Surface duplicate keys in staging tables, measure column-level null-rate anomalies against historical baselines, and detect day-over-day volume drift.
  • Integrity & completeness validation: Check referential integrity (orphan fact rows), cross-table row-count reconciliation, and missing partitions/completeness coverage.
  • StarRocks-specific health checks: Validate tablet replication/health signals, compaction backlog indicators, and routine load error states.
  • Operational DQ scanning: Run the checks via a Python DQ scan class and store results for trend monitoring.

Quick Start

Ask the agent to validate yesterday’s StarRocks data in sales.orders by running freshness, duplicate, null-rate, volume drift, and partition completeness checks and then saving the pass/fail metrics into monitoring.dq_results.

Frequently Asked Questions about starrocks-data-quality-guardian

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect duplicate keys and null anomalies in StarRocks staging tables?▼

Detect StarRocks duplicate keys and null-rate anomalies by running SQL-based scanning checks that measure column-level null rates against historical baselines and surface duplicate rows in staging tables.

What SQL checks can I run to monitor StarRocks data freshness and volume drift?▼

Monitor StarRocks data freshness and volume drift by running SQL checks that compare MAX(updated_at) against SLAs and measure day-over-day volume drift for fact and dimension tables.

How do I validate referential integrity and partition completeness in StarRocks?▼

Validate StarRocks referential integrity and partition completeness by executing SQL joins to detect orphan fact rows and checking for missing partitions to ensure full completeness coverage.

Can I check StarRocks tablet replication health and routine load error states?▼

Check StarRocks tablet replication health, compaction backlog indicators, and routine load pause states by running engine health validation SQL queries to catch storage and loading issues.

Do I need Python to run StarRocks data quality monitoring checks?▼

Running StarRocks data quality monitoring requires optional Python-based scanning via a DQ scan class, requiring the pymysql dependency, alongside StarRocks-compatible SQL validation patterns.

How do I persist StarRocks data quality results for trend monitoring?▼

Persist StarRocks data quality results by saving pass/fail metrics from freshness, duplicate, null-rate, and volume drift checks into a dedicated DQ monitoring table for trend analysis.

Related Skills

Explore More Data & Analytics Skills→
nq-rdlnq-rdl

starrocks

Guide StarRocks table design, partitioning, and materialized view management.

Official
Advanced
pluginagentmarketplacepluginagentmarketplace
5

monitoring-observability

Implement Prometheus metrics, Grafana dashboards, structured logging, and alerting for data pipelines.

Community
Advanced
ivanshamaevivanshamaev
14

starrocks-self-healing

Restart transiently paused StarRocks Routine Load jobs and remediate ingestion health issues.

Community
Advanced
timequitytimequity
6

data-quality

Detect data quality issues in dbt and Great Expectations pipelines.

Official
Advanced