perf-gate

Validates performance optimization claims on shared Ascend NPU machines using same-window A/B measurement protocols.

14|5|Updated Mar 30, 2026
One-click install
npx skills add https://github.com/Ascend/MindIE-SD --skill perf-gate-ascend
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: perf-gate
Source: https://github.com/Ascend/MindIE-SD/tree/main/.agents/skills/perf-gate
Command: npx skills add https://github.com/Ascend/MindIE-SD --skill perf-gate-ascend

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? On shared multi-card Ascend NPU machines, performance numbers drift across measurement windows, so naive comparisons produce false conclusions about whether an optimization actually helps. This Skill defines the acceptance criteria that decide whether a measured speedup is real and which numbers are allowed into the summary report. ## Core Features & Use Cases - Two-state discipline: Separates exploratory readings (single-run, few-step, must be tagged [探索], never enter the summary table) from acceptance-state results (same-window control arm, repeated hot runs, full evidence) that may be written into reports. - Same-window A/B authority: Enforces that only same-window differences count as gains, with A/B/A drift correction when differences fall inside the noise floor, plus explicit noise-floor thresholds for when to declare "indistinguishable". - Triple-evidence rule: Every reported number requires HTTP 200 status, artifact byte count, and a server-side log line; md5 plus key-tensor dump dual evidence for lossless claims, with fidelity comparison against an independent ground-truth implementation to close the md5 blind spot. - Use Case: A teammate reports the new decoder made inference slower (15.005s vs 14.585s yesterday). Use this Skill to reject the cross-window subtraction, decompose the difference into drift versus structural cost, and require a same-window A/B or A/B/A run before any conclusion enters the report. ## Quick Start Ask the AI to verify whether a specific optimization's measured speedup on the shared NPU machine is real and eligible for the summary report.

Frequently Asked Questions about perf-gate

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I verify a performance optimization actually works on a shared NPU machine?

Run a same-window A/B comparison with a control arm in the same measurement window, since cross-window absolute values are not comparable due to drift. If the difference falls inside the noise floor, use A/B/A correction (base, arm, base) and report only the corrected net difference.

Why did my inference get slower after a code change?

Cross-window drift on shared machines often exceeds the real effect of an optimization, so subtracting numbers from different windows is invalid. Decompose the difference using each window's own stage accounting to separate forward drift from structural cost before concluding anything.

Does matching md5 prove my optimization is lossless?

Matching md5 only proves the optimization did not change the output, not that the output is correct. If all arms share a defective implementation, the gate passes while results are wrong, so add a fidelity comparison against an independent ground-truth implementation using MAE and Pearson correlation.

Can I average three runs taken hours apart for benchmarking?

No. The three runs must be hot requests from the same window and same run, with cold starts recorded separately. Averaging across windows measures machine drift rather than performance and produces meaningless numbers.

Why is my curl timing much higher than the server-side forward time?

Log timings from curl get polluted by profiler sampling, request queuing, and neighbor load on shared machines. Cross-validate against server-side stage accounting and request cadence, and when they conflict, trust the server-side numbers and exclude sampled requests from averages.

When can exploratory benchmark numbers go into the summary report?

Never directly. Exploratory readings (single-run, few-step, no control arm) must be tagged as exploratory and stay out of the main table; promotion to acceptance state requires a same-window control arm, the required repetition count, and full triple evidence.