skills.rest
Giskard-AI
Giskard-AIOFFICIAL
Explore all skills by this author →
WebsiteGitHub

rag-evaluator

Evaluate RAG systems for groundedness, relevance, and citation accuracy.

5|Updated Apr 13, 2026
Search Tags:#rag#evaluation#ai-testing#groundedness#giskard
One-click install
npx skills add https://github.com/Giskard-AI/giskard-skills --skill rag-evaluator
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: rag-evaluator
Source: https://github.com/Giskard-AI/giskard-skills/tree/main/oss/checks/rag-evaluator
Command: npx skills add https://github.com/Giskard-AI/giskard-skills --skill rag-evaluator

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires giskard-checks, giskard.agents, giskard.agents.generators, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill helps you evaluate the quality of RAG (Retrieval-Augmented Generation) systems, ensuring groundedness, relevance, and overall quality of answers.

Core Features & Use Cases

  • Groundedness Evaluation: Check if answers are supported by provided context.
  • Answer Relevance: Ensure answers address the question correctly.
  • Out-of-Scope Refusal: Verify that the system declines when it cannot answer.
  • Retrieval Quality: Assess the quality of the retrieval system if one is exposed.
  • Citation Accuracy: Check if citations are accurate and support the claims made.
  • Use Case: When you have a RAG system and want to ensure it is providing high-quality, grounded answers that are relevant to the user's query.

Quick Start

Use the rag-evaluator skill to generate an evaluation suite for your RAG system. Provide information about your agent, KB, and any relevant data.

Frequently Asked Questions about rag-evaluator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate groundedness and answer relevance in my RAG system?▼

To evaluate groundedness and answer relevance in a RAG system, you can generate an evaluation suite that checks if answers are supported by provided context and address the user's question correctly.

Can I check citation accuracy and retrieval quality if my retriever is exposed?▼

Yes, you can check citation accuracy if citations are used, and the evaluation handles retrieval quality automatically when your retriever is exposed to the testing suite.

How do I verify out-of-scope refusal for a retrieval-augmented generation agent?▼

To verify out-of-scope refusal for a retrieval-augmented generation agent, the evaluation suite tests whether the system properly declines to answer when it lacks the necessary context.

Do I need the giskard-checks library to run RAG evaluation tests?▼

Yes, you need the giskard-checks library installed because the RAG evaluation relies on its built-in checks to measure groundedness, relevance, and overall answer quality.

What is the best way to test if a RAG system provides high-quality answers?▼

The best way to test if a RAG system provides high-quality answers is to provide your agent, knowledge base, and relevant data to an evaluation suite that checks groundedness and relevance.

What are the limitations of evaluating a RAG system without an exposed retriever?▼

Without an exposed retriever, the evaluation is limited to checking answer groundedness, relevance, and out-of-scope refusal, skipping the direct assessment of retrieval quality.

Related Skills

Explore More Software Engineering Skills→
floflo777floflo777
33

rag-eval

Evaluate RAG systems using retrieval, generation, and latency metrics.

Community
Intermediate
marchattonmarchatton
5

evaluate-rag

Evaluate RAG pipelines with Recall@k, Precision@k, MRR, and NDCG@k metrics.

Community
Advanced
Giskard-AIGiskard-AI
5

scenario-generator

Generate adversarial test scenarios and suites for AI agents using giskard.checks.

Official
Advanced
BagelHoleBagelHole
46

rag-observability-evals

Monitor RAG systems with retrieval and generation quality metrics.

Community
Advanced