ai-evaluation-suite

LLM output evaluation uses automated scoring on coherence, relevance, and factuality to detect hallucinations and inconsistencies. This Skill provides LLM-as-judge patterns, BLEU/ROUGE/Perplexity metrics, and hallucination detection to catch quality issues before deployment, ensuring your model performs reliably in production.

Yes, this Skill evaluates RAG systems and supports standard benchmarks like MMLU and HumanEval. It provides F1 scoring, LLM-as-judge assessment, and integration-ready outputs for comparing retrieval quality and model performance across real-world deployment scenarios.

Track token usage, latency, and cost metrics to compare model efficiency. This Skill provides cost and performance optimization tracking, enabling you to identify cheaper alternatives or faster variants while maintaining quality before going live.

This Skill automatically identifies demographic biases and factual inconsistencies in AI-generated content using bias detection patterns. It integrates with evaluation frameworks to flag bias issues early, supporting safety and fairness requirements across deployment contexts.

Yes, use this Skill to score and compare LLM outputs across different prompts and models. Functional metrics like BLEU, ROUGE, and F1 enable systematic evaluation of prompt variations and side-by-side model comparison to optimize performance.

This Skill depends on Anthropic, NumPy, and pytest. No external components are required; setup integrates directly with your LLM pipeline, allowing immediate evaluation of outputs without additional infrastructure.