calibrate

Post-launch monitoring identifies error patterns by cataloging production failures, root causes, and fixes. This Skill provides a structured template to triage and track AI feature errors, ensuring systematic documentation for continuous improvement.

Reviewing evaluation effectiveness requires coverage and gap analysis to ensure evals surface real production issues. This Skill assesses whether existing evals evolve with new failure modes and accurately detect post-launch regressions.

Deciding on agency promotion requires a checklist-based verdict evaluating quality, safety, monitoring, and operational readiness. This Skill guides promotion decisioning to prevent premature autonomy and reduce regressions in AI-driven product features.

You can run quick health checks by requesting a summary of error patterns, eval gaps, and promotion recommendations for a specific feature. This Skill supports weekly health checks, monthly eval reviews, and quarterly deep calibration cycles.

Perform eval gap analysis when your AI feature encounters new failure modes or when preparing for agency promotion. This Skill helps determine if your current evals adequately cover production issues and evolve alongside post-launch error patterns.