skills.rest
rblake2320
rblake2320COMMUNITY
Explore all skills by this author →
GitHub

rag-perf

Benchmarks NVIDIA Blueprint servers with per-stage Rapid RAG performance analysis.

Updated May 29, 2026
Search Tags:#performance#benchmarking#yaml#latency#throughput#aiperf#rag-perf
One-click install
npx skills add https://github.com/rblake2320/vigil --skill rag-perf-rblake2320
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: rag-perf
Source: https://github.com/rblake2320/vigil/tree/main/.claude/skills/rag-perf
Command: npx skills add https://github.com/rblake2320/vigil --skill rag-perf-rblake2320

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Provides a repeatable, config-driven mechanism to benchmark the performance of a deployed NVIDIA RAG Blueprint server, quantifying latency, throughput, and bottlenecks to inform optimization and capacity planning.

Core Features & Use Cases

  • End-to-end profiling: captures server-side timings across retrieval, reranking, LLM inference, and overall TTFT.
  • Load testing with presets: uses aiperf and YAML configurations to simulate realistic traffic and measure throughput and error rates.
  • Bottleneck diagnosis: reports a stage-by-stage breakdown and highlights whether retrieval, reranking, or LLM time dominates.
  • Reproducibility: supports grid sweeps and saved configurations so runs are repeatable across environments.

Quick Start

Run a quick benchmark by executing uv run --project scripts/rag-perf rag-perf -c scripts/rag-perf/configs/single_run.yaml

Frequently Asked Questions about rag-perf

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark NVIDIA RAG performance end-to-end?▼

Benchmark NVIDIA RAG performance by running configurable YAML presets against a deployed RAG server to capture latency, throughput, and stage-by-stage timings across retrieval, reranking, and LLM inference stages.

How do I measure RAG latency and identify bottlenecks in retrieval or LLM inference?▼

Measure RAG latency and identify bottlenecks using per-point profiling that provides a stage-by-stage breakdown of server-side timings, highlighting whether retrieval, reranking, or LLM time dominates overall performance.

Can I load test a RAG server with configurable concurrency and top-k parameters?▼

Load test a RAG server with configurable concurrency and top-k parameters using YAML presets and aiperf to simulate realistic traffic, measure throughput, and track error rates under controlled workloads.

What metrics are captured when profiling RAG performance?▼

Profiling RAG performance captures metrics including Time To First Token (TTFT), overall throughput, error rates, and detailed server-side execution timings across retrieval, reranking, and LLM inference stages.

Do I need a remote RAG server to run reproducible performance benchmarks?▼

Reproducible performance benchmarks can be driven against either a self-hosted or remote RAG server, using YAML configurations and grid sweeps to ensure repeatable runs across different environments.

Why does my RAG throughput drop under high concurrency?▼

RAG throughput drops under high concurrency are diagnosed by stage-by-stage bottleneck reporting, which isolates whether retrieval, reranking, or LLM inference time dominates and causes performance degradation.

Related Skills

Explore More Software Engineering Skills→
sayalinvidiasayalinvidia

rag-blueprint

Deploy NVIDIA RAG Blueprint across self-hosted, NVIDIA-hosted, or retrieval-only modes.

Community
Advanced
mtsatryanmtsatryan
1

performance-engineer

Profile CPU, memory, I/O, and latency to identify performance bottlenecks.

Community
Advanced
vecearvecear

load-test-scenario-builder

Design and execute k6 load tests for web applications and APIs.

Community
Advanced
webrix-aiwebrix-ai
1

performance-monitoring

Set up performance monitoring with dashboards, alerts, and SLO tracking.

Official
Advanced