review-log

To identify stuck hypotheses from run logs, you can analyze runs/<id>/log.md to detect recurring hypotheses across rounds, evaluate narrative coherence, and flag overfitting signals. This structured review process highlights hypotheses that fail to progress across experiment iterations.

Overfitting signal detection in run tracking flags improvements on training metrics that fail to transfer to holdout data. By analyzing run narratives in log.md, it surfaces these discrepancies to help researchers avoid models that memorize training data without generalizing.

To generate a structured review from a markdown experiment log, run the review-log command on your run directory. It reads runs/<id>/log.md, extracts coherence and overfitting signals, and writes actionable feedback with recommended next steps to runs/<id>/review.md.

You should run a log review for experiment evaluation after long experiments spanning 10 or more rounds. Applying the review process at this stage ensures enough data exists to accurately detect narrative incoherence, stuck hypotheses, and overfitting signals from the run logs.

The review-log workflow does not require external dependencies to parse run logs. It operates independently to read runs/<id>/log.md, extract evaluation metrics, and write structured feedback to review.md without needing additional components or packages installed.