Compares production and test data streams using MetricDiff to validate consistency.
npx skills add https://github.com/huangcd/gcusage --skill data-validation-huangcd
Please help me install this Agent Skill. Skill: data-validation Source: https://github.com/huangcd/gcusage/tree/main/.github/skills/data-validation Command: npx skills add https://github.com/huangcd/gcusage --skill data-validation-huangcd
DataValidation pipelines help compare production vs test data streams using MetricDiff, enabling automated validation and optional ClickHouse export for SQL querying.
Use DataValidationBuilder to define and deploy a validation, then query results in ClickHouse.
High-intent search queries and answers about installing and using this skill.
Automating prod vs test data validation involves comparing production and test data streams using MetricDiff to verify data consistency. You can validate pipeline outputs by calculating count, countDistinct, and sum metrics across specified dimensions.
MetricDiff is used in production and test data comparison to calculate dimension-level metric differences including count, countDistinct, and sum. It enables automated validation of data consistency between prod and test streams.
Setting up DataValidationBuilder with constructStreamRead requires defining proper inputs including prodPath and testPath. You then create a data validation config to compare prod vs test streams and proceed with deployment steps.
Yes, you can optionally export validation results to ClickHouse via DORA for SQL querying. This enables you to query and validate MetricDiff outputs directly in ClickHouse after comparing your prod and test streams.
Yes, data validation supports dimension-level changes between prod and test streams. You can compare streams across specified dimensions and obtain MetricDiff results for count, countDistinct, and sum to identify variations.
Comparing prod vs test data streams using DataValidationBuilder requires proper inputs including prodPath and testPath. You need these paths to construct stream reads and create a valid data validation configuration.
durable-streamsCreate stream forks referencing source data up to a specified offset.
ClawLink-HQExecute SQL-like queries on ClickHouse databases via HTTP API without writing SQL.
tomsiwikCreate, transform, and consume Effect-TS Streams for lazy data processing.
patrykkopycinskiGenerate Kibana dashboards with metric, XY, gauge, datatable, and markdown panels.
