accuracy-gate

Validates whether code optimizations preserve numerical results using layered equivalence criteria and staged acceptance tests.

14|5|Updated Mar 30, 2026
One-click install
npx skills add https://github.com/Ascend/MindIE-SD --skill accuracy-gate-ascend
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: accuracy-gate
Source: https://github.com/Ascend/MindIE-SD/tree/main/.agents/skills/accuracy-gate
Command: npx skills add https://github.com/Ascend/MindIE-SD --skill accuracy-gate-ascend

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, and includes scripts (resource) and references (resource) components.

What problem does it solve? When you replace an operator, fuse kernels, change parallel partitioning, or alter compilation paths, you need to prove the change did not alter results. This Skill provides a rigorous acceptance standard that distinguishes bit-exact equivalence from approximate similarity, preventing silent numerical regressions from being shipped as "lossless" optimizations. ## Core Features & Use Cases - Layered Equivalence Definition: Classifies changes as L1 bit-exact (torch.equal per shape), L2 numerical gate (explicit thresholds with justification), or L3 lossy gate (quality metrics plus visual review), forbidding mixing of levels. - Three-Stage Acceptance Sequence: Enforces same-config rerun bit-exactness, cross-config numerical gates with md5 artifact comparison, and quality gates for lossy tiers, with mandatory evidence records. - Silent Failure Localization: Provides a systematic debugging workflow (CPU ground truth, per-layer divergence bisection, operator isolation, threshold sweeps) for wrong results with no errors or NaNs. - Use Case: You rewrote a deconvolution that takes 97% of decode time and the output md5 changed with max difference 9.5e-7. The Skill tells you this is not lossless, requires explicit L2 downgrade declaration or rejection, and directs you to check index/phase/order bugs rather than precision. ## Quick Start Ask the AI to verify whether your operator replacement is truly lossless and what acceptance evidence you need to collect.

Frequently Asked Questions about accuracy-gate

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I prove an operator replacement is bit-exact?

Embed per-shape torch.equal assertions directly in the replacement implementation so mismatches raise errors on first call. Verify every shape that occurs in production, not just one, since equivalence is often shape-dependent. Then run the three-stage acceptance: same-config rerun, cross-config md5 comparison, and quality gate if lossy.

Is a max difference of 1e-6 considered lossless?

No, any non-zero difference fails L1 bit-exact equivalence and cannot be delivered as lossless. You must explicitly downgrade to L2 with a documented threshold and justification, or reject the change if the output md5 changed and gains fall within noise.

Why is my rewritten operator output wrong but shows no errors?

Rewrite errors almost always come from index, phase, or ordering mistakes rather than numerical precision. Check grouping dimensions, stride and dilation correspondence, padding placement, channel order, and accumulation order before adjusting dtype or precision.

How do I debug wrong NPU results when CPU runs correctly?

Use the silent failure localization workflow: establish CPU fp32 ground truth, instrument per-layer outputs to find the first divergence point, isolate the suspect operator, and sweep input sizes to distinguish fixed-index defects from size thresholds. The layer_bisect.py and op_threshold_sweep.py scripts provide ready-made skeletons.

When is the quality gate required for acceptance?

The quality gate (level 3) applies only to lossy tiers such as quantization, sparsity, caching, or weight replacement. Lossless changes like equivalent replacement or fusion only need bit-exact rerun and md5 comparison; they must not use lossy-tier criteria to pass.

What are the limitations of equivalence acceptance testing?

Conclusions expire when hardware, drivers, framework versions, shapes, dtypes, weights, or parallel topology change, requiring retesting. Equivalence also does not guarantee usability: a bit-exact rewrite can still be rejected if it causes OOM or removes kernel overlap.