What problem does it solve? Mechanistic interpretability experiments can pass an honest evaluation audit while the underlying mechanism is mis-extracted, mis-scaled, or mis-applied — for example, a steering coefficient that was never swept, or a large alpha whose apparent effect is actually out-of-distribution collapse. This Skill audits the mechanism rigor itself, per claim, so downstream readers do not update on artifacts of under-tuned interventions. ## Core Features & Use Cases - Per-Claim Mechanism-Rigor Verdict: Scopes the audit to one claim's milestones, scripts, and outputs, then returns a PASS/WARN/FAIL/N/A verdict on that claim's mechanism rigor. - Steering Coefficient Sweep Check (Check A): Verifies that additive activation interventions (steering, CAA, DAS, RepE, SAE feature scaling, activation patching, ROME) used a swept alpha grid, logged capability metrics, identified a usable plateau, and locked alpha mid-plateau, with checks B–F reserved for future rigor dimensions. - Independent Cross-Model Review: The executor only collects claim-scoped artifacts; an external LLM reviewer via the llm-chat MCP reads the code and judges rigor, keeping the executor out of the verdict. - Use Case: After running a steering-vector experiment for claim C1, invoke the audit on refine-logs/ with -- claim C1 to produce MECHANISM_AUDIT.md and MECHANISM_AUDIT.json, which a caller like /auto-verify combines with the methodology audit via max-severity. ## Quick Start Run /mechanism-audit on your experiment directory with -- claim C1 and an optional -- output-dir to receive a per-claim mechanism-rigor verdict report.
