skills.rest
NVIDIA
NVIDIAOFFICIAL
Explore all skills by this author →
WebsiteGitHub

trt-cpp-runtime-quickstart

Load and run TensorRT engines with the modern C++ Runtime API.

13.2k|2.4k|Updated May 2, 2019
Search Tags:#deployment#runtime#cpp#inference#tensorrt
One-click install
npx skills add https://github.com/NVIDIA/TensorRT --skill trt-cpp-runtime-quickstart
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: trt-cpp-runtime-quickstart
Source: https://github.com/NVIDIA/TensorRT/tree/main/.agents/skills/trt-cpp-runtime-quickstart
Command: npx skills add https://github.com/NVIDIA/TensorRT --skill trt-cpp-runtime-quickstart

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill unit solves the problem of loading and running TensorRT engines from C++ without using deprecated APIs, providing a minimal and modern approach to C++ deployment.

Core Features & Use Cases

  • Modern Runtime API: Utilizes the latest TensorRT 11/10.x Runtime API for efficient engine deployment.
  • Minimal Example: Provides a clean, minimal example for quick deployment harness creation.
  • Dynamic Shape Handling: Offers guidance on handling dynamic shapes in inference.

Quick Start

Run the TensorRT engine 'model.engine' with the command: ./run model.engine

Frequently Asked Questions about trt-cpp-runtime-quickstart

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I load and run a TensorRT engine in C++ without using deprecated binding-index APIs?▼

You can load and run a TensorRT engine in C++ using the modern Runtime API, which bypasses deprecated binding-index methods. This approach provides a minimal deployment harness for the latest TensorRT 10.x and 11.x versions.

What is the modern way to handle dynamic shapes during TensorRT inference in C++?▼

Handling dynamic shapes during TensorRT inference involves using the modern C++ Runtime API, which offers specific guidance for managing varying input dimensions. This ensures efficient engine execution without relying on deprecated binding indices.

Can I use the TensorRT C++ Runtime API to deploy my model on servers with specific CUDA versions?▼

Yes, you can deploy model engines using the TensorRT C++ Runtime API, provided your environment has matching CUDA and TensorRT versions. It requires a compiled .plan or .engine file to run successfully.

What do I need to run a pre-compiled TensorRT .engine file using C++?▼

To run a pre-compiled TensorRT .engine file, you need the modern C++ Runtime API, a matching CUDA environment, and the correct TensorRT version. You can execute the engine using a simple command like ./run model.engine.

Does this TensorRT C++ deployment approach support older TensorRT versions?▼

This TensorRT C++ deployment approach targets the modern Runtime API available in TensorRT 10.x and 11.x. It specifically avoids deprecated binding-index APIs found in older versions to ensure a minimal and updated deployment.

Why should I use the modern TensorRT C++ Runtime API instead of older binding-index methods?▼

Using the modern TensorRT C++ Runtime API avoids deprecated binding-index methods, providing a cleaner and more efficient way to load engines. It offers a minimal example for quick deployment harness creation and dynamic shape handling.

Related Skills

Explore More Software Engineering Skills→
nishide-devnishide-dev

ml-model-export

Export trained PyTorch models to ONNX, TorchScript, and TensorRT formats.

Community
Advanced
NVIDIA-OmniverseNVIDIA-Omniverse
201

reading-render-output

Map Ovrtx render outputs to accessible pixel data on CPU or CUDA.

Official
Intermediate
martialarconmartialarcon
1

jetson-context

Provide Jetson Orin Nano hardware and software context for AI/ML optimization.

Community
Intermediate
jayll1303jayll1303
17

tensorrt-llm

Convert HuggingFace checkpoints into optimized TensorRT-LLM engines with quantization.

Community
Advanced