eval-analysis

What problem does it solve?

This Skill analyzes eval results to understand signal quality and guide prompt/config changes while enforcing anti-bias safeguards.

Core Features & Use Cases

• Gather recent eval data with eval_runs to identify run IDs, timestamps, and included scenarios.
• Pull observations via eval_observations for detailed per-scenario signal data and metadata.
• Drill into affected scenarios with eval_scenario_detail to inspect inputs, scoring criteria, and notes, then read relevant prompts and config sources.
• Apply anti-bias gating rules to propose safe, targeted changes and document counter-scenarios.

Quick Start

Analyze the latest eval_run data to surface signal quality and guide prompt/config adjustments.