skills.rest
rapidsai
rapidsaiOFFICIAL
Explore all skills by this author →
Website𝕏@RAPIDSaiGitHub

perf-compare-cudf

Benchmarks libcudf NVBench suites on a branch or PR against main and reports performance differences.

9.7k|1.1k|Updated May 7, 2017
Search Tags:#benchmarking#regression testing#GPU#Performance#CUDA#cuDF#nvbench
One-click install
npx skills add https://github.com/rapidsai/cudf --skill perf-compare-cudf
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: perf-compare-cudf
Source: https://github.com/rapidsai/cudf/tree/main/.agents/skills/perf-compare-cudf
Command: npx skills add https://github.com/rapidsai/cudf --skill perf-compare-cudf

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It automates the tedious process of verifying whether cuDF code changes cause performance regressions or improvements by running identical NVBench benchmarks on both a target branch (WIP or PR) and the main branch, then producing a structured comparison report.

Core Features & Use Cases

  • Automated A/B Benchmarking: Builds and runs the same libcudf NVBench benchmarks on a target branch and on main, with identical benchmark and axis options on both sides.
  • PR and WIP Support: Checks out a cuDF PR via the gh CLI or benchmarks current uncommitted changes, handling stashing, merging with main, and restoring the original git state afterward.
  • Structured Comparison Report: Uses nvbench_compare.py with a 5% threshold to generate a COMPARISON.md report including hardware details, branch SHAs, axis coverage, and per-suite metric tables.
  • Use Case: A cuDF contributor modifies a CUDA kernel and wants to confirm no regression before opening a PR; the skill runs the affected benchmark suites on both branches and summarizes meaningful timing deltas.

Quick Start

Compare the libcudf benchmark performance of my current branch against main and summarize any regressions.

Frequently Asked Questions about perf-compare-cudf

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compare cuDF benchmark performance between a branch and main?▼

Build libcudf benchmarks on your target branch with BUILD_BENCHMARKS=ON, run the selected NVBench suites, then repeat on a clean checkout of main. Use nvbench_compare.py with a threshold to diff the JSON results and identify meaningful timing changes.

How do I benchmark a cuDF pull request before merging?▼

Check out the PR with gh pr checkout, merge it with the latest main if behind, build and run the relevant NVBench benchmarks, then repeat on main. Compare the two result directories to detect regressions or improvements.

What tools are required to run libcudf NVBench benchmarks?▼

You need the cuDF devcontainer environment, CMake configured with BUILD_BENCHMARKS=ON, an NVIDIA GPU visible via nvidia-smi, and the gh CLI authenticated for PR targets. Benchmark binaries are produced under cpp/build/latest/benchmarks.

Why does NVBench segfault at the end of a benchmark suite?▼

An end-of-suite segfault after results are written is a known benign behavior and can be ignored if the JSON output was saved. If a specific config throws an exception, verify both branches behave identically before excluding it from the comparison.

How do I avoid noisy GPU benchmark results?▼

Pick an idle GPU with nvidia-smi before every run and pin execution with CUDA_VISIBLE_DEVICES and -d 0. Re-run surprising failures once, and treat deltas as significant only when they exceed 5% and the noise level of both sides.

Related Skills

Explore More Software Engineering Skills→
rapidsairapidsai
9.7k

build-test-cudf

Build and test code changes inside a cudf devcontainer.

Official
Intermediate
spiriMirrorspiriMirror
307

gpu-optimization

Profile, analyze, and optimize CUDA kernels in LibuIPC using Nsight Compute.

Official
Advanced
mitchdennymitchdenny
173

surface-benchmarker

Run and interpret Hex1b Surface API benchmarks with BenchmarkDotNet and dotnet run.

Community
Basic
Spectaculous-CodeSpectaculous-Code

admin-panel-builder

Create and refactor React admin pages with Supabase for the KR92 Bible Voice project.

Official
Intermediate