experiment-analyzer

Experiment analysis evaluates completed experiment data against baseline metrics to classify outcomes as Win, Loss, Inconclusive, or Neutral. Provide results, baseline, and statistical significance; the analyzer computes change percentages, validates your initial hypotheses, and generates a markdown report with actionable insights and 2-3 follow-up experiment proposals.

Hypothesis validation checks whether your initial predictions held true by comparing expected outcomes against actual results. It reveals which assumptions were correct, which were wrong, and informs whether to iterate on the approach or pivot direction entirely.

Yes. Experiment analysis supports comparing multiple completed experiments side-by-side, classifying each outcome, and calculating aggregate success rates. This reveals patterns across your growth cycle and identifies which experiment types or hypotheses perform best.

Provide completed experiment data in JSON format, including actual results, baseline metrics, and statistical significance values. The analyzer validates the structure, processes the metrics, and generates insights from properly formatted experiment records.

Analysis reports are automatically archived as markdown files in the experiments/archive/ directory, named [experiment-id]_analysis.md. This ensures your insights and follow-up recommendations persist and remain accessible for future reference.

Inconclusive outcomes indicate insufficient statistical significance to draw firm conclusions; the data doesn't prove the hypothesis either way. Neutral outcomes show no meaningful change between treatment and baseline. Both require different follow-up actions.