skills.rest
lofibrainwav
lofibrainwavCOMMUNITY
Explore all skills by this author →
Website𝕏@jada_kinig_6onGitHub

ragas-evaluator

Score RAG output fidelity and relevance using four metrics.

Updated Aug 23, 2026
Search Tags:#recall#rag#evaluation#precision#faithfulness#relevancy
One-click install
npx skills add https://github.com/lofibrainwav/AFO_Kingdom --skill ragas-evaluator
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ragas-evaluator
Source: https://github.com/lofibrainwav/AFO_Kingdom/tree/main/skills/ragas-evaluator
Command: npx skills add https://github.com/lofibrainwav/AFO_Kingdom --skill ragas-evaluator

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides a rigorous evaluation framework for RAG outputs across faithfulness, relevancy, precision, and recall.

Core Features & Use Cases

  • Faithfulness: Ensure answers align with retrieved context.
  • Relevancy & Precision: Validate that responses address the question with relevant data.
  • Context Recall: Verify that retrieved context covers the needed information.

Quick Start

scores = await evaluator.evaluate(question="What is Trinity Score?", answer="Trinity Score ...", contexts=["Trinity Score measures Truth, Goodness..."])

Frequently Asked Questions about ragas-evaluator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate RAG system quality across faithfulness and relevance?▼

RAG evaluation measures whether retrieved-answer pairs are faithful to source context and relevant to questions. This Skill scores faithfulness, answer relevancy, context precision, and context recall—four metrics that validate retrieval and generation fidelity in QA and knowledge pipelines.

What metrics does a RAG evaluator use to measure retrieval performance?▼

Context recall measures whether retrieved documents contain needed information; context precision measures whether retrieved results are relevant to the question. Together they balance completeness against noise in retrieval-augmented systems.

How do I validate that my RAG answers are grounded in retrieved context?▼

Faithfulness scoring ensures answers align only with facts present in retrieved context, preventing hallucination. Pass question, answer, and context documents to produce a structured evaluation score.

Can I use RAG evaluation for document-grounded question answering benchmarks?▼

Yes. This Skill applies to QA benchmarks, knowledge retrieval systems, and document-grounded reasoning pipelines where answers must be validated against retrieved context using recall, precision, and relevancy metrics.

What input data do I need to score RAG system outputs?▼

Provide a question, generated answer, and retrieved context documents. The evaluator ingests these three inputs and returns four metric scores: faithfulness, answer relevancy, context precision, and context recall.

Related Skills

Explore More Data & Analytics Skills→
Dianshu-LiaoDianshu-Liao
2

rag-implementation

Build retrieval-augmented generation systems with citation-ready outputs from external documents.

Community
Intermediate
MantisWareMantisWare

eval-rag

Evaluate RAG pipelines with retrieval and generation metrics and bottleneck analysis.

Community
Advanced
thejordanleopoldthejordanleopold
4

rag-and-vector-search

Implement retrieval-augmented generation pipelines with vector search.

Community
Advanced
pluginagentmarketplacepluginagentmarketplace
2

evaluation-metrics

Evaluate LLM outputs using BLEU, ROUGE, BERTScore, and RAG metrics.

Community
Advanced