Orchestrate filesystem-based RAG benchmarking with corpus/train.json and evaluate_rag.py.
npx skills add https://github.com/rblake2320/vigil --skill rag-eval-rblake2320
Please help me install this Agent Skill. Skill: rag-eval Source: https://github.com/rblake2320/vigil/tree/main/.claude/skills/rag-eval Command: npx skills add https://github.com/rblake2320/vigil --skill rag-eval-rblake2320
Filesystem-based RAG benchmarks require a reproducible layout and a consistent evaluation workflow to measure retrieval quality and grounding in a realistic setting.
From the repository root, run the evaluation by invoking uv run --project scripts/eval python scripts/eval/evaluate_rag.py with your dataset-paths, host, and port to execute a RAGAS benchmark.
High-intent search queries and answers about installing and using this skill.
To benchmark RAG quality, you can structure your documents under a corpus/ directory and use a train.json file to run RAGAS evaluations. This validates your local dataset layout and measures retrieval and grounding metrics.
Evaluating RAG retrieval and grounding accuracy is best done by orchestrating a RAGAS benchmark against your configured RAG stack. This measures how well your system retrieves relevant context and grounds responses in local datasets.
Yes, you need a configured RAG stack accessible at localhost:8081 or 8082. The evaluation script queries this server to measure retrieval quality, grounding, and efficiency metrics against your local datasets.
Run a RAGAS benchmark by executing uv run --project scripts/eval python scripts/eval/evaluate_rag.py from the repository root. You must provide your dataset paths, host, and port to generate quality metrics.
Yes, you can measure RAG efficiency across different ingestors and local datasets. The evaluation workflow orchestrates benchmarking against your RAG server to compare ingestion strategies and retrieval performance.
Inconsistent RAG benchmark results often stem from unvalidated corpus/train.json layouts. Preparing and validating your filesystem-based dataset structure ensures a reproducible layout and consistent evaluation workflow.
sayalinvidiaBenchmark NVIDIA RAG Blueprint servers with config-driven profiling and aiperf load tests.
MuvonDesign and troubleshoot production RAG pipelines with chunking, hybrid retrieval, reranking, and evaluation.
AgentientConfigure Vertex AI RAG pipelines for grounding agent outputs in private data sources.
ContextJet-aiAdd offline and online evaluation scoring to LLM and agent applications.
