rag-perf

Benchmark NVIDIA RAG Blueprint servers with config-driven profiling and aiperf load tests.

Updated Apr 16, 2026
One-click install
npx skills add https://github.com/sayalinvidia/sayali-skills-test --skill rag-perf
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: rag-perf
Source: https://github.com/sayalinvidia/sayali-skills-test/tree/main/skills/rag-perf
Command: npx skills add https://github.com/sayalinvidia/sayali-skills-test --skill rag-perf

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Benchmark a deployed NVIDIA RAG Blueprint server by driving profiling data and optional aiperf load tests via a config-driven CLI.

Core Features & Use Cases

  • Config-driven profiling and load-testing against a live RAG server to quantify latency, throughput, and bottlenecks.
  • Supports grid sweeps over key knobs like concurrency, vdb_top_k, and reranker_top_k, producing reproducible results and detailed reports.
  • Useful for iteration on retrieval/reranker tuning, capacity planning, and SLA verification with repeatable YAML presets.

Quick Start

Run rag-perf with a YAML config to measure performance against your RAG server and generate a summarized report of TTFT and throughput.

Frequently Asked Questions about rag-perf

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark RAG performance and measure TTFT for a deployed server?

Benchmark RAG performance by running config-driven profiling against a live RAG server to measure latency metrics like TTFT and throughput. Use YAML presets to drive load tests and generate summarized reports for retrieval tuning.

What is the best way to run load tests on an NVIDIA RAG Blueprint server?

Run load tests on an NVIDIA RAG Blueprint server by using a CLI to drive optional aiperf load tests and profiling data. This config-driven approach quantifies bottlenecks across concurrency, vdb_top_k, and reranker_top_k knobs.

How can I sweep concurrency and reranker_top_k configurations to find RAG bottlenecks?

Sweep concurrency and reranker_top_k configurations by running grid sweeps via YAML presets against your RAG server. This identifies bottlenecks and produces reproducible results for capacity planning and SLA verification.

What prerequisites are needed to run RAG latency benchmarks with aiperf?

To run RAG latency benchmarks, you need a local RAG server running on port 8081, uv installed, Python 3.11+, and the rag-perf package installed in a virtual environment to execute the profiling scripts.

Can I use synthetic data for RAG throughput benchmarking?

You can use synthetic data or file-based inputs for RAG throughput benchmarking. The config-driven CLI supports both input types to measure performance metrics and generate detailed reports.

Why does my RAG benchmarking test require YAML presets?

RAG benchmarking tests require YAML presets to ensure reproducible results when sweeping configurations like vdb_top_k and concurrency. These presets define the testing parameters for consistent latency and throughput measurement.