experiment-audit

An experiment integrity audit examines evaluation scripts, result files, and configurations to detect fake ground truth, self-normalized score fraud, and phantom metrics by cross-verifying artifacts with an independent reviewer model, generating PASS/WARN/FAIL findings.

Validating evaluation scripts after an ML training run requires restricting the executor to artifact path collection while an independent reviewer model fully inspects files to verify evaluation scope, detect dead code paths, and confirm research transparency.

Self-normalized score fraud in machine learning evaluation occurs when reported metrics are manipulated through improper normalization; auditing experimental artifacts detects this by cross-verifying evaluation scripts and result files against intended pipeline outputs.

Yes, you can check for missing result-file claims and dead evaluation code paths before publication by running an experimental audit that scans artifacts and outputs evidence-linked PASS/WARN/FAIL verdicts to machine-readable JSON and Markdown formats.

Experiment integrity auditing uses a split workflow where the executor side only collects artifact paths without reading contents, while the reviewer side performs full file inspection to judge research credibility across ML training and evaluation workflows.

An experimental audit outputs findings to EXPERIMENT_AUDIT.md and EXPERIMENT_AUDIT.json, providing both human-readable reports and machine-readable files for downstream integrity tagging of PASS/WARN/FAIL verdicts and paper-narrative claims.