skills.rest
netdata
netdataOFFICIAL
Explore all skills by this author →
WebsiteGitHub

project-query-corpus

Validate Netdata query engine correctness using an end-to-end black-box testing suite.

80.4k|6.6k|Updated Jun 17, 2013
Search Tags:#regression#Testing#correctness#query-engine#GO#Netdata
One-click install
npx skills add https://github.com/netdata/netdata --skill project-query-corpus
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: project-query-corpus
Source: https://github.com/netdata/netdata/tree/main/.agents/skills/project-query-corpus
Command: npx skills add https://github.com/netdata/netdata --skill project-query-corpus

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the difficulty of verifying the correctness of complex time-series query engines by providing a rigorous, oracle-based testing framework that prevents regressions and ensures semantic consistency.

Core Features & Use Cases

  • First-Principles Oracles: Validates engine output against mathematical laws like conservation, additivity, and monotonicity.
  • Source-Derived Ports: Implements engine algorithms as independent oracles to verify complex transforms without relying on engine output.
  • Use Case: When developing a fix for a query-engine bug, use this suite to run the corpus, verify the fix against existing contracts, and ensure no collateral regressions occur in the query logic.

Quick Start

Run the full query corpus test suite from the tests/query-corpus directory using the go test command.

Frequently Asked Questions about project-query-corpus

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run the query corpus test suite?▼

Navigate to the tests/query-corpus directory and execute the go test ./... -count=1 command. Ensure the Netdata daemon is built beforehand using ninja to provide the necessary binary for the harness.

What is the difference between Class A and Class B oracles?▼

Class A oracles derive expectations from first-principles math like conservation and additivity. Class B oracles are ports of engine algorithms written from source code, used when transforms are design decisions rather than derivable math.

Can I use the corpus to test custom query engine branches?▼

Yes, you can validate a fix branch by setting the QUERY_CORPUS_NETDATA and QUERY_CORPUS_SRC environment variables to point to your custom build and source paths before running the test suite.

Why does the test suite fail when I change the engine output?▼

The suite uses strict byte-pins and contract assertions to detect output changes. If you intentionally change the output format, you must justify the contract change and update the pins accordingly.

What should I do if a test case fails?▼

A failing test indicates a broken contract. You must determine if the engine is incorrect and requires a fix, or if the behavior is intended and the test case needs to be updated with a documented ruling.

Related Skills

Explore More Software Engineering Skills→
projectbluefinprojectbluefin
134

packaging-pre-built-binaries

Package pre-built static binaries into a build system with multi-architecture support.

Official
Intermediate
WebWakaHubWebWakaHub
20

Test Results Analyzer

Analyze test execution results to identify failure patterns and release readiness risks.

Official
Advanced
ohyesgocoolohyesgocool

resolve-conflicts

Merge target branch changes into a feature branch with conflict resolution and verification.

Community
Advanced
wojonswojons
4

testing-unit

Execute unit tests across programming languages and testing frameworks.

Community
Basic