What problem does it solve? Building a reliable agent requires knowing whether it actually works, and ad-hoc manual testing cannot catch regressions, hallucinations, or broken tool usage at scale. This Skill provides a structured evaluation methodology — the Quality Flywheel — for running agents over datasets, grading traces with built-in or custom metrics, analyzing failures, and iterating on fixes. ## Core Features & Use Cases - Eval execution loop: Run agents-cli eval run (or decoupled eval generate + eval grade) to execute an agent over a JSON EvaluationDataset and produce timestamped JSON/HTML score reports. - Metrics selection and custom metrics: Choose from built-in metrics like multi_turn_task_success, final_response_quality, hallucination, and safety, or define custom LLM-as-judge (prompt_template) and Python (custom_function) metrics in eval_config.yaml. - Failure analysis and optimization: Use eval compare to verify fixes without regressions, eval analyze for LLM-based failure clustering, and eval optimize for GEPA prompt tuning. - Use Case: After scaffolding an ADK agent, run agents-cli eval run against tests/eval/datasets/basic-dataset.json, inspect the failing multi_turn_tool_use_quality scores in the HTML report, fix the tool descriptions, and confirm improvement with eval compare. ## Quick Start Ask the assistant to run an evaluation of your agent with agents-cli eval run and show the scores table for every case.
