calibration-review

Calibration review analyzes vault snapshots, learned heuristics, and database outcomes to measure prediction agent health using Brier score and win rate. It identifies biases and generates actionable recommendations for adjusting thresholds, persona weights, and heuristic strategies to improve prediction accuracy.

Analyzing Brier score and win rate thresholds requires consolidating vault snapshots and predictions database outcomes to surface health metrics. Domain-aware reviews identify biases and opportunities, generating structured recommendations for adjusting threshold values and refining heuristic strategies.

Run a meta-analysis for heuristic and persona weight adjustments during weekly reviews or domain-specific audits. The calibration review evaluates learned heuristics against database outcomes to determine if persona weights require tuning or if new heuristics should be applied.

Yes, vault snapshots provide the calibration state needed to identify prediction biases and calibrate decision policies. The review process reads snapshot data alongside learned heuristics and database outcomes to perform domain-aware analyses and output recommended next steps.

Calibration review operates without external dependencies, using vault snapshots, learned heuristics, and the predictions database internally. It extracts Brier score and win rate metrics directly from these sources to support governance and decision-making with structured summaries.