investigating-eval-results

What problem does it solve?

This skill solves the critical problem of AI coding agents failing web platform evaluation tests, helping teams quickly identify root causes of discrepancies between guided and unguided performance to deliver reliable, modern web implementation results.

Core Features & Use Cases

• Systematic 7-Step Investigation Checklist: A mandatory, structured workflow that covers prompt auditing, eval execution, grader validation, guidance coverage checks, and trajectory tracing to eliminate guesswork.
• False Negative & True Negative Resolution: Built-in protocols to distinguish between overly rigid graders and unclear guidance content, with concrete fixes for both failure types.
• Autonomous Iteration & Progress Tracking: Supports looping through fix, re-run, and measure cycles until 100% guided pass rates are achieved, with required 30-second progress check-ins that include log references for transparency. Use case: For teams running AI coding agent evals for modern web use cases, use this skill to autonomously debug failures, fix prompts/guidance/graders, and consistently hit 100% guided pass rates without manual issue tracking.

Quick Start

Use the investigating-eval-results skill to diagnose why the animate-dialog-modal use case is failing its guided eval runs and implement a fix to reach 100% pass rate.