Convert ONNX models to TensorRT engines and verify numerical parity.
npx skills add https://github.com/NVIDIA/TensorRT --skill trt-onnx-quickstart
Please help me install this Agent Skill. Skill: trt-onnx-quickstart Source: https://github.com/NVIDIA/TensorRT/tree/main/.agents/skills/trt-onnx-quickstart Command: npx skills add https://github.com/NVIDIA/TensorRT --skill trt-onnx-quickstart
This Skill enables developers to convert ONNX models to TensorRT, ensuring efficient and accurate inference on NVIDIA GPUs.
Run the trt-onnx-quickstart skill with the Hugging Face model ID or ONNX file to generate a TensorRT engine and verify its numerical parity against ONNX Runtime.
High-intent search queries and answers about installing and using this skill.
To convert ONNX models to TensorRT for GPU inference, you can use a conversion process that builds optimized engines and verifies numerical parity against ONNX Runtime. This requires an NVIDIA GPU, Python, and Polygraphy.
TensorRT supports dynamic shapes and multi-input ONNX models by handling variations in batch size and sequence length. The conversion process builds engines that accommodate these dimensions during inference.
You verify numerical accuracy during ONNX to TensorRT conversion by performing a numerical parity check. This ensures the generated TensorRT engine matches ONNX Runtime outputs within specified tolerances.
An NVIDIA GPU is required to run TensorRT engines, as TensorRT is specifically designed to optimize and execute inference on NVIDIA hardware. You also need TensorRT and Python installed.
The best way to optimize ONNX inference performance is converting the model to a TensorRT engine. This process leverages NVIDIA GPU hardware capabilities for high-performance execution while maintaining output accuracy.
Andrew-GirgisTrain and fine-tune language and vision models on Hugging Face Jobs.
jayll1303Convert HuggingFace checkpoints into optimized TensorRT-LLM engines with quantization.
NVIDIA-NeMoVerify weight parity between HF and MCore model conversions using Python scripts.
nishide-devExport trained PyTorch models to ONNX, TorchScript, and TensorRT formats.
