data-validation

Compares production and test data streams using MetricDiff to validate consistency.

Updated Apr 8, 2026

Search Tags:#clickhouse #dora #datavalidation #metricdiff #data-validation-builder #prod-test

Ready to run

npx skills add https://github.com/huangcd/gcusage --skill data-validation-huangcd

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: data-validation
Source: https://github.com/huangcd/gcusage/tree/main/.github/skills/data-validation
Command: npx skills add https://github.com/huangcd/gcusage --skill data-validation-huangcd

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

DataValidation pipelines help compare production vs test data streams using MetricDiff, enabling automated validation and optional ClickHouse export for SQL querying.

Core Features & Use Cases

DataValidationBuilder config creation with constructStreamRead usage
Compare prod vs test streams across dimensions and obtain MetricDiff (count, countDistinct, sum)
Optional export to ClickHouse via DORA for SQL querying and validation

Quick Start

Use DataValidationBuilder to define and deploy a validation, then query results in ClickHouse.

Frequently Asked Questions about data-validation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate prod vs test data validation for pipeline outputs?▼

Automating prod vs test data validation involves comparing production and test data streams using MetricDiff to verify data consistency. You can validate pipeline outputs by calculating count, countDistinct, and sum metrics across specified dimensions.

What is MetricDiff used for in production and test data comparison?▼

MetricDiff is used in production and test data comparison to calculate dimension-level metric differences including count, countDistinct, and sum. It enables automated validation of data consistency between prod and test streams.

How do I set up DataValidationBuilder with constructStreamRead?▼

Setting up DataValidationBuilder with constructStreamRead requires defining proper inputs including prodPath and testPath. You then create a data validation config to compare prod vs test streams and proceed with deployment steps.

Can I export validation results to ClickHouse via DORA for SQL querying?▼

Yes, you can optionally export validation results to ClickHouse via DORA for SQL querying. This enables you to query and validate MetricDiff outputs directly in ClickHouse after comparing your prod and test streams.

Does data validation support dimension-level changes between prod and test streams?▼

Yes, data validation supports dimension-level changes between prod and test streams. You can compare streams across specified dimensions and obtain MetricDiff results for count, countDistinct, and sum to identify variations.

What inputs are required to compare prod vs test data streams using DataValidationBuilder?▼

Comparing prod vs test data streams using DataValidationBuilder requires proper inputs including prodPath and testPath. You need these paths to construct stream reads and create a valid data validation configuration.