skills.rest
applied-artificial-intelligence
applied-artificial-intelligenceOFFICIAL
Explore all skills by this author →
WebsiteGitHub

llm-evaluation

Evaluate LLM applications with automated metrics and LLM-as-judge scoring.

81|22|Updated Nov 23, 2025
Search Tags:#LLM#RAG#benchmark#hallucination#quality-metrics#prompt-testing#evaluate
One-click install
npx skills add https://github.com/applied-artificial-intelligence/claude-code-toolkit --skill llm-evaluation-applied-artificial-intelligence
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llm-evaluation
Source: https://github.com/applied-artificial-intelligence/claude-code-toolkit/tree/main/skills/llm-evaluation
Command: npx skills add https://github.com/applied-artificial-intelligence/claude-code-toolkit --skill llm-evaluation-applied-artificial-intelligence

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides a comprehensive framework for evaluating and testing LLM applications, including prompt testing, hallucination detection, benchmarking, and quality metrics.

Core Features & Use Cases

  • Automated metrics: BLEU, ROUGE, BERTScore, perplexity, and other quantitative measures to assess output quality.
  • LLM-as-judge & rubrics: Rubric-based scoring, binary pass/fail tests, and human-in-the-loop evaluation.
  • End-to-end pipelines: Evaluation across retrieval, grounding, and answer quality, including RAG workflows.
  • Use cases: Validate prompts, compare model variants (A/B), and establish CI/CD evaluation for LLM deployments.

Quick Start

Run a quick evaluation by supplying a simple prompt and ground-truth answer, then review both automated metrics and rubric scores.

Frequently Asked Questions about llm-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate RAG retrieval quality and grounding?▼

To evaluate RAG retrieval quality and grounding, run end-to-end evaluation pipelines that assess retrieval accuracy and perform grounding checks to ensure generated answers are factually supported by retrieved context.

What is the best way to detect hallucinations in LLM outputs?▼

Hallucination detection is best handled by combining automated quantitative metrics with LLM-as-judge rubric scoring to systematically identify ungrounded or fabricated information in generated text.

How do I set up A/B testing to compare different LLM model variants?▼

Set up A/B testing for LLM model variants by applying automated metrics like BLEU and BERTScore alongside rubric-based pass/fail tests to quantitatively compare output quality across different prompts or models.

Can I use automated metrics for prompt testing and regression checks?▼

Yes, use automated metrics such as ROUGE, perplexity, and BLEU for prompt testing and regression checks to establish CI/CD evaluation pipelines and ensure LLM application quality remains stable across changes.

Does LLM-as-judge scoring work with human evaluation rubrics?▼

LLM-as-judge scoring integrates with human-in-the-loop evaluation rubrics, supporting binary pass/fail tests and structured scoring to combine automated assessment with human oversight for reliable outcomes.

Related Skills

Explore More Data & Analytics Skills→
floflo777floflo777
33

rag-eval

Evaluate RAG systems using retrieval, generation, and latency metrics.

Community
Intermediate
illogicalillogical

evaluation

Assess agent performance with multi-dimensional rubrics and weighted scores.

Community
Advanced
ContextJet-aiContextJet-ai
29

check-answer-consistency

Detect potential misinformation in LLM answers by measuring consistency across samples.

Official
Basic
GoogleCloudPlatformGoogleCloudPlatform
9

run-gcsfuse-npi

Orchestrate GCSFuse validation and benchmarking pipelines across GCE and GKE environments.

Official
Advanced