experiment-audit

Audit experimental claims with statistical, baseline, and ablation checks.

Updated Aug 23, 2026

Search Tags:#audit #statistics #claims #experiments #ablation #baselines #review-guidelines

Ready to run

npx skills add https://github.com/ZhangHanbo/alpha_research --skill experiment-audit-zhanghanbo

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: experiment-audit
Source: https://github.com/ZhangHanbo/alpha_research/tree/main/skills/experiment-audit
Command: npx skills add https://github.com/ZhangHanbo/alpha_research --skill experiment-audit-zhanghanbo

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Audit experimental claims for rigor by verifying statistical sufficiency, baselines, and ablations; detect overclaiming and identify missing baselines.

Core Features & Use Cases

Deterministic statistical audit against experiment directories or papers under review.
Baseline strength assessment and ablation isolation checks.
Automated generation of a structured audit report for review.

Quick Start

Run the audit against an experiment directory containing results.json, trials.csv, or metrics.json to generate a structured audit report.

Frequently Asked Questions about experiment-audit

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I audit experiment statistics for rigor and baseline strength?▼

To audit experiment statistics for rigor, you run a deterministic statistical audit against an experiment directory or paper to verify statistical sufficiency, assess baseline strength, and check ablation isolation, generating a structured audit record.

What is ablation isolation checking in experimental claims?▼

Ablation isolation checking is a statistical audit process that verifies whether experimental results genuinely isolate the contribution of individual components, preventing overclaiming by validating ablation study boundaries against provided metrics.

Can I use this to detect overclaiming and missing baselines in papers under review?▼

Yes, you can detect overclaiming and identify missing baselines by applying the statistical audit and baseline strength assessment to papers under review, which maps directly to standard review vectors and generates a structured audit report.

How do I run an audit on an experiment directory with results.json or trials.csv files?▼

You run the audit by targeting an experiment directory that contains results.json, trials.csv, or metrics.json files, which triggers the scripts/audit_stats.py module to compute thresholds and generate a structured audit record.

Does the audit require any external dependencies or specific frameworks to function?▼

No external dependencies or specific frameworks are required to function, as the audit relies solely on internal scripts to perform the deterministic statistical audit, baseline strength assessment, and ablation isolation checks against your provided files.