experiment-analyze

Reproducibility analysis compares observed outcomes against pre-registered hypotheses and benchmarks to compute per-condition verdicts and assign statuses of pass, partial, or fail. It operates across reproduction, diagnostic, and approach modes to detect failure modes.

You need results.jsonl, config.yaml, and scripts/audit_stats.py stored under the experiments/<exp_id> directory. The audit also requires formalization.md and benchmarks.md to compare outcomes against pre-registered hypotheses and generate structured findings.

Yes, this skill automates post-experiment analysis by synthesizing a structured finding that surfaces reproducibility issues and proposes backward triggers like t4, t5, t7, t8, t14, or t15. It appends experiment_analysis and finding records while marking the output for human review.

Mode-aware analysis is used when you need to compute per-condition verdicts, check mechanisms like ablations versus priors, and determine hypothesis status across reproduction, diagnostic, or approach modes. It ensures accurate failure mode detection based on the specific experimental context.

Yes, while the skill appends experiment_analysis and finding records and proposes backward triggers for state transitions, it requires explicit human confirmation before any state transition occurs. This governed action ensures provenance and human oversight.