skills.rest
NVIDIA
NVIDIAOFFICIAL
Explore all skills by this author →
WebsiteGitHub

trtllm-case-executor

Executes TensorRT-LLM tests, benchmarks, and evaluations locally or via Slurm job submission.

14.5k|2.7k|Updated Aug 16, 2023
Search Tags:#docker#hpc#tensorRT-llm#Test automation#Slurm#benchmark execution#gpu testing
One-click install
npx skills add https://github.com/NVIDIA/TensorRT-LLM --skill trtllm-case-executor
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: trtllm-case-executor
Source: https://github.com/NVIDIA/TensorRT-LLM/tree/main/.claude/skills/trtllm-case-executor
Command: npx skills add https://github.com/NVIDIA/TensorRT-LLM --skill trtllm-case-executor

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Running TensorRT-LLM test cases, benchmarks, and evaluations requires choosing the right execution environment (local GPU, Docker, or a Slurm cluster), resolving container images, sizing GPU/node counts, and validating Slurm accounts and partitions — a manual, error-prone process this Skill automates.

Core Features & Use Cases

  • Four workflow dispatch: Classifies and runs custom commands, pre-built trtllm-bench commands, trtllm-eval commands, or pytest/perf-sanity test commands without modifying them.
  • Environment-aware routing: Detects local GPU/Docker availability or routes to local/remote Slurm clusters, including SSH-based remote execution with MFA handling.
  • Automatic resource sizing: Parses tensor/pipeline parallelism flags or perf-sanity YAML configs to derive GPU counts, per-node devices, and node counts.
  • Slurm environment validation: Runs detect_slurm_env.sh to resolve and validate accounts, partitions, per-partition hardware (arch, GRES, GPUs per node), and PMIx plugins before job submission.
  • Use Case: A performance engineer needs to run a perf-sanity test for a GPT-OSS model on 8 B200 GPUs. Provide the pytest command and perf config YAML, and the Skill sizes the job, validates the Slurm partition, builds the submission script, and monitors the run to completion.

Quick Start

Run the TensorRT-LLM benchmark command 'trtllm-bench --model llama throughput --tp 4' on the appropriate GPU environment and report the results.

Frequently Asked Questions about trtllm-case-executor

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run TensorRT-LLM tests on a Slurm cluster?▼

Provide a pre-built test command (pytest, trtllm-bench, trtllm-eval, or custom) and the Skill detects the environment, validates your Slurm account and partition via detect_slurm_env.sh, builds the submission script, and dispatches the job. Remote clusters are reached over SSH with MFA handling.

How do I run trtllm-bench or trtllm-eval commands with this executor?▼

Pass the fully built command string as bench_cmd or eval_cmd. The Skill parses --tp/--pp or --tp_size/--pp_size only to determine required GPU count and never modifies the command itself.

Does the executor support multi-node GPU jobs?▼

Yes. For perf-sanity tests with a perf_config_yaml, it reads hardware.gpus_per_node and tensor/pipeline parallel sizes to compute total devices and node_count as ceil(total/per-node). Bench and eval workflows are single-node only.

Can I run TensorRT-LLM tests without Docker or Slurm?▼

Yes. When Docker is unavailable, the local_direct scenario runs the command in place with a timeout wrapper, background hang detection on the log file, and exit-code-based result mapping.

What happens if my Slurm partition or account is invalid?▼

The detection script lists your valid accounts and partitions; if your supplied values don't match, the Skill surfaces the mismatch with available options and asks before proceeding. It never silently falls back to defaults.

Why does the executor not build my test commands?▼

Command construction is intentionally handled upstream by the caller (e.g., a test-specialist skill using build_test_command.py). This Skill only accepts pre-built command strings, keeping execution and command generation as separate concerns.

Related Skills

Explore More Software Engineering Skills→
ZardLi1115ZardLi1115
4

tensorrt-llm

Compile and serve LLMs on NVIDIA GPUs with TensorRT-LLM.

Community
Intermediate
AGMO-IncAGMO-Inc
4

benchmark

Execute benchmark commands and store latency and throughput metrics in Obsidian vaults.

Official
Advanced
NVIDIA-TAONVIDIA-TAO
83

tao-run-on-slurm

Automate remote TAO GPU job submission on SLURM clusters via SSH.

Official
Advanced
nanocoainanocoai
30.5k

claw

Send command line prompts to NanoClaw agent containers by group, JID, or session ID.

Official
Basic