Diagnose ADK evaluation failures and recommend fixes for agent results.
npx skills add https://github.com/SeanChenR/maestro-agent --skill adk-eval-guide
Please help me install this Agent Skill. Skill: adk-eval-guide Source: https://github.com/SeanChenR/maestro-agent/tree/main/.agents/skills/adk-eval-guide Command: npx skills add https://github.com/SeanChenR/maestro-agent --skill adk-eval-guide
This guide reduces guesswork when ADK evaluation metrics fail by explaining evaluation metrics, evalset schemas, judge-model patterns, and common failure modes so engineers can diagnose, fix, and iterate on agent quality reliably.
Run a focused evaluation using your evalset, inspect per-invocation metric breakdowns, and apply the recommended fixes to prompts, evalsets, or config until scores stabilize.
High-intent search queries and answers about installing and using this skill.
ADK agent evaluation metric failures are fixed by inspecting tool trajectories and LLM-as-judge outputs to diagnose common failure modes. Apply the iterative eval-fix loop to update prompts, evalsets, or eval_config.json until scores stabilize.
An LLM-as-judge rubric metric in ADK uses a model to score agent responses against defined criteria. Configure deterministic or rubric-based judge settings in eval_config.json for reproducible CI and local testing.
Configure eval_config.json for multimodal and user-simulation scenarios by defining the evalset schema and applying session overrides. This wires custom metrics and judge model options to test agent interactions reliably.
ADK evalset non-determinism in CI often stems from google_search behavior or improper judge settings. Stabilize scores by applying practical fixes to config patterns and choosing deterministic or rubric-based judge configurations.
Yes, ADK evaluation supports tool trajectory scoring via the tool_trajectory_avg_score metric. It evaluates agent tool usage patterns alongside final_response_match_v2 and hallucinations_v1 metrics for comprehensive quality assessment.
yu13130122297Identify uncertainties in research framing, datasets, metrics, and baselines.
daymadeConfigure and run Promptfoo LLM evaluation workflows with YAML configs and CLI commands.
hq-opensourceEvaluate ADK-based agents using metrics, evalsets, and LLM-as-judge.
SimHackerCreate weighted rubrics and score outputs against defined criteria.
