skills.rest
jpoindexter
jpoindexterCOMMUNITY
Explore all skills by this author →
Website𝕏@microninkGitHub

llm-evals-and-retrieval-quality

Evaluate and improve LLM and RAG systems with retrieval metrics and grounding checks.

1|Updated Jun 9, 2026
Search Tags:#llm evaluation#ai engineering#rag systems#evaluation metrics#retrieval quality
One-click install
npx skills add https://github.com/jpoindexter/design-and-ai-skills --skill llm-evals-and-retrieval-quality
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llm-evals-and-retrieval-quality
Source: https://github.com/jpoindexter/design-and-ai-skills/tree/main/ai-engineering-skills/llm-evals-and-retrieval-quality
Command: npx skills add https://github.com/jpoindexter/design-and-ai-skills --skill llm-evals-and-retrieval-quality

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides a comprehensive guide to evaluating and improving the quality of LLM and RAG systems, ensuring robust performance and accurate retrieval.

Core Features & Use Cases

  • LLM Evaluation: Offers detailed guidance on various evaluation sets, including golden, regression, adversarial, unit, and end-to-end evaluations.
  • Retrieval Metrics: Delivers insights into retrieval metrics like recall@k, MRR, nDCG, and context-level RAG metrics.
  • Grounding and Faithfulness: Focuses on grounding, faithfulness, and attribution to ensure the accuracy of generated content.
  • Eval Set Construction: Provides guidance on building and maintaining high-quality eval sets for continuous improvement.
  • Process Integration: Integrates evaluation processes into development workflows for effective quality control.

Quick Start

Load the llm-evals-and-retrieval-quality skill and follow the comprehensive guide to evaluate your LLM and RAG systems.

Frequently Asked Questions about llm-evals-and-retrieval-quality

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate retrieval quality in RAG systems?▼

Evaluate RAG retrieval quality using metrics like recall@k, MRR, and nDCG. This Skill guides you through calculating context-level metrics to measure retrieval accuracy and rank retrieved documents effectively.

What is the best way to check LLM grounding and faithfulness?▼

Check LLM grounding by verifying attribution and faithfulness to source context. This Skill provides detailed guidance on ensuring generated content remains accurate and grounded.

How do I build an evaluation set for LLM regression testing?▼

Build an LLM evaluation set by constructing golden, adversarial, and unit test cases. This Skill outlines how to create and maintain high-quality eval sets for continuous regression testing.

Can I integrate LLM evaluation metrics into development workflows?▼

Yes, you can integrate LLM evaluation into development workflows. This Skill explains how to embed evaluation processes into AI engineering pipelines for effective quality control.

What types of evaluation sets are needed for comprehensive LLM evaluation?▼

Comprehensive LLM evaluation requires golden, regression, adversarial, unit, and end-to-end evaluation sets. This Skill details how to leverage each type to measure system performance.

Related Skills

ArogyaReddyArogyaReddy

llm-evaluation

Automates LLM output evaluation with metrics, human feedback, and benchmarking.

Community
Advanced
hamelsmuhamelsmu
1.6k

evaluate-rag

Evaluate RAG pipelines by measuring retrieval metrics and generation quality.

Community
Advanced
devCharuzudevCharuzu

rag-architect

Design and deploy production-grade RAG pipelines with chunking, embeddings, vector stores, and evaluation.

Community
Advanced
JhabbigJhabbig

llm-evaluation

Design evaluation frameworks for LLM applications with automated and human scoring.

Community
Advanced