harness-summary

Multi-sprint evaluation data analysis aggregates pass rates, pass@k, and pass^k metrics to identify performance trends and recurring failure patterns for continuous system improvement. It processes evaluation reports, configs, and contracts to detect anomalies across sprints.

Cross-sprint evaluation metric aggregation calculates overall pass rates and detects anomalies across multiple evaluation cycles in AI workflows. It synthesizes historical evaluation reports to provide comprehensive insights into system reliability and consistency.

Generate a sprint review summary by analyzing evaluation reports, configs, and contracts to calculate pass rates and detect failure patterns. This produces insights on consistency and regressions to guide next development cycles.

Yes, recurring failure patterns and regressions are detected by analyzing multi-sprint evaluation data during project retrospectives. The analysis identifies performance improvements or degradations over time and highlights bottlenecks needing attention.

Yes, evaluation configs and contracts are required to track system stability improvements accurately. They provide the necessary baseline and criteria to synthesize multi-sprint results and determine if system reliability is improving.