skills.rest
GoogleChrome
GoogleChromeOFFICIAL
Explore all skills by this author →
GitHub

investigating-eval-results

Diagnose root causes of low pass rates in AI coding agent web evaluation tests.

926|71|Updated Jan 27, 2026
Search Tags:#prompt validation#agent trajectory#ai coding agents#eval investigation#guided pass rate#grader calibration#web guidance
One-click install
npx skills add https://github.com/GoogleChrome/modern-web-guidance-src --skill investigating-eval-results
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: investigating-eval-results
Source: https://github.com/GoogleChrome/modern-web-guidance-src/tree/main/.agents/skills/eval-investigation
Command: npx skills add https://github.com/GoogleChrome/modern-web-guidance-src --skill investigating-eval-results

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill solves the critical problem of AI coding agents failing web platform evaluation tests, helping teams quickly identify root causes of discrepancies between guided and unguided performance to deliver reliable, modern web implementation results.

Core Features & Use Cases

  • Systematic 7-Step Investigation Checklist: A mandatory, structured workflow that covers prompt auditing, eval execution, grader validation, guidance coverage checks, and trajectory tracing to eliminate guesswork.
  • False Negative & True Negative Resolution: Built-in protocols to distinguish between overly rigid graders and unclear guidance content, with concrete fixes for both failure types.
  • Autonomous Iteration & Progress Tracking: Supports looping through fix, re-run, and measure cycles until 100% guided pass rates are achieved, with required 30-second progress check-ins that include log references for transparency. Use case: For teams running AI coding agent evals for modern web use cases, use this skill to autonomously debug failures, fix prompts/guidance/graders, and consistently hit 100% guided pass rates without manual issue tracking.

Quick Start

Use the investigating-eval-results skill to diagnose why the animate-dialog-modal use case is failing its guided eval runs and implement a fix to reach 100% pass rate.

Frequently Asked Questions about investigating-eval-results

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I investigate AI coding agent eval failures for web platform use cases?▼

AI coding agent eval failures are diagnosed by comparing guided versus unguided performance, validating grader tolerance, and checking reference guidance content coverage. This structured investigation distinguishes between overly rigid graders and unclear guidance content.

How do I fix low pass rates for AI coding agent evaluation tests?▼

Low pass rates are fixed by autonomously iterating through fix, re-run, and measure cycles with 30-second progress check-ins. Loop through prompt fixes, guidance updates, and grader calibrations until achieving the 100% guided pass rate target.

How do I distinguish between false negatives and true negatives in agent eval trajectories?▼

False negatives and true negatives are distinguished by applying built-in protocols to validate grader tolerance and audit prompt non-prescriptiveness. This reveals whether failures stem from overly rigid graders or unclear guidance content.

What is grader calibration for web guidance implementations?▼

Grader calibration for web guidance implementations is the process of validating grader tolerance against reference guidance content coverage. It ensures graders accurately measure modern web implementation results without being overly rigid.

Why do my guided eval runs fail to reach 100% pass rate for modern web use cases?▼

Guided eval runs fail to reach 100% pass rate due to discrepancies between guided and unguided performance, overly rigid graders, or gaps in reference guidance content coverage. Trace agent trajectories to locate the exact failure point.

Can I autonomously debug AI coding agent eval failures without manual issue tracking?▼

Yes, you can autonomously debug AI coding agent eval failures without manual issue tracking. The workflow supports looping through fixes, re-runs, and measurements with required 30-second progress check-ins that include log references for transparency.

Related Skills

Explore More Software Engineering Skills→
KanakMalpaniKanakMalpani

eval-harness

Define and run eval-driven evaluations with pass@k and pass^k metrics.

Community
Advanced
caobingshengcaobingsheng
1

ai-product

Design, validate, test, and ship scalable prompt-driven features with safety checks.

Community
Advanced
amo-tech-aiamo-tech-ai

testing-workflow

Automate layer-by-layer end-to-end testing workflows with SQL, API, component, and Playwright tests.

Community
Intermediate
robit-manrobit-man

iteration-control

Track retry attempts and escalate failed validation loops to users.

Community
Advanced