What problem does it solve? Writing and running evaluations for AI agents and skills requires learning the AgentEvals EVAL.yaml schema, configuring targets, authoring LLM judges, and wiring results into CI pipelines. This Skill provides the operational knowledge to do all of that with the AgentV CLI without memorizing the specification. ## Core Features & Use Cases - EVAL.yaml Authoring: Write declarative evaluation files with test cases, deterministic assertions, rubrics, and LLM/code graders, including conversion from existing evals.json manifests. - Target Configuration & Comparison: Configure .agentv/targets.yaml for OpenAI, Anthropic, CLI, or HTTP agents, then run the same suite against multiple targets and diff results with agentv compare. - CI/CD Integration & Debugging: Gate pull requests with --threshold and JUnit XML output, and debug failing llm-grader scores by inspecting JSONL run results. - Use Case: You have a skill with an evals/evals.json file and want CI to fail when eval pass rate drops below 80%. This Skill walks you through converting to EVAL.yaml, setting a target, and adding a GitHub Actions step with agentv eval --threshold 0.8. ## Quick Start Ask the agent to create an EVAL.yaml evaluation file for your skill and show the agentv CLI command to run it in CI.
