What problem does it solve? Testing AI and LLM outputs is hard because responses are non-deterministic and can hallucinate, leak data, show bias, or ignore instructions. This Skill provides a structured framework to plan, implement, and grade AI output evaluations with measurable metrics instead of subjective judgment. ## Core Features & Use Cases - Nine evaluation metrics: Four OWASP LLM security metrics (prompt injection, sensitive info disclosure, excessive agency, unbounded consumption) plus five quality metrics (hallucination, instruction following, bias, confidence score, custom rubric). - Provider-agnostic adapter pattern: Swap between Claude, OpenAI, or custom LLM backends via configuration without changing test code, using the @evalCase decorator that extends existing @atc tests. - Layered cost model: Run free mocked checks on pull requests, cheap cached grading in CI, and full LLM evaluation for nightly or production sampling. - Use Case: A team shipping a RAG chatbot uses the skill to verify every response cites sources faithfully, resists prompt injection, and scores above 70% on a custom faithfulness rubric before release. ## Quick Start Ask the AI to plan and implement an evaluation for your chatbot using the genai-testing skill, starting with hallucination and instruction-following metrics.
