skills.rest
NVIDIA
NVIDIAOFFICIAL
Explore all skills by this author →
WebsiteGitHub

tao-train-nvpanoptix3d

Train, evaluate, export, and run inference for TAO NVPanoptix3D panoptic 3D reconstruction models.

3.2k|370|Updated Feb 25, 2026
Search Tags:#model training#docker#3D reconstruction#autoML#tao toolkit#panoptic segmentation#nvpanoptix3d
One-click install
npx skills add https://github.com/NVIDIA/skills --skill tao-train-nvpanoptix3d
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: tao-train-nvpanoptix3d
Source: https://github.com/NVIDIA/skills/tree/main/skills/tao-train-nvpanoptix3d
Command: npx skills add https://github.com/NVIDIA/skills --skill tao-train-nvpanoptix3d

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Training and deploying NVPanoptix3D models for panoptic 3D scene reconstruction requires navigating complex TAO Toolkit specs, Docker-based workflows, dataset path conventions, and AutoML routing. This Skill guides an AI agent through the correct commands, spec overrides, and error handling so users avoid misconfigured runs and wasted GPU time.

Core Features & Use Cases

  • Docker-native action commands: Provides ready-to-adapt docker run commands for train, evaluate, inference, and export actions using the TAO PyTorch container image.
  • Spec and dataset guidance: Documents per-action dataset requirements (frustum masks, label maps, JSON splits), mandatory spec overrides, and important parameters like model.sem_seg_head.num_classes and train.precision.
  • AutoML train routing: Routes train actions through the TAO AutoML runner with train_loss as the optimization metric, including handling of multi-fidelity resume jobs.
  • Use Case: A robotics engineer asks the agent to train an NVPanoptix3D model on a Matterport3D dataset; the Skill supplies the correct downsample factor, dataset paths, GPU settings, and launch command.

Quick Start

Ask your agent to train an NVPanoptix3D model on your posed RGB image dataset using the TAO Toolkit Docker image.

Frequently Asked Questions about tao-train-nvpanoptix3d

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train an NVPanoptix3D model with TAO Toolkit?▼

Run the train action inside the TAO PyTorch Docker image using `python -m nvidia_tao_pytorch.cv.nvpanoptix3d.entrypoint.nvpanoptix3d train -e /specs/train.yaml`. Mount your data, specs, and results directories, and set mandatory dataset spec overrides such as frustum_mask_path and label_map.

What datasets does NVPanoptix3D support for 3D reconstruction?▼

NVPanoptix3D supports front3d, matterport, synthetic_hospital, and synthetic_warehouse datasets. Use downsample_factor 1 for Front3D and synthetic data, and 2 for Matterport3D.

Why does NVPanoptix3D training fail with fp16 precision?▼

The current training entrypoint only supports fp32 and raises 'Only fp32 precision is supported' for fp16, even though the schema advertises it. Set `train.precision: fp32` for train and resume runs.

Why is the nvpanoptix3d command not found in the TAO PyTorch image?▼

The 7.0 PyTorch image contains the NVPanoptix3D package but no console script. Invoke it via the module entrypoint: `python -m nvidia_tao_pytorch.cv.nvpanoptix3d.entrypoint.nvpanoptix3d <action> -e <spec>`.

What GPUs are required for NVPanoptix3D training?▼

A minimum of 2 GPUs is required with 4 recommended, each with 40GB or more VRAM such as A100. 3D reconstruction is memory intensive, and only DDP (not FSDP) is supported for distributed training.

Why is the 3D ONNX file missing after NVPanoptix3D export?▼

The current export entrypoint only exports the 2D model component to `export.onnx_file_2d`. The `export.onnx_file_3d` field exists in the schema but is not produced by this toolkit image.

Related Skills

Explore More Software Engineering Skills→
NVIDIA-TAONVIDIA-TAO
83

tao-launch-workflow

Automate TAO workflow launches with platform selection, credentials, and preflight checks.

Official
Advanced
sixscripts-aisixscripts-ai

computer-vision-expert

Design end-to-end computer vision pipelines with YOLO26, SAM 3, and VLMs.

Community
Advanced
NVIDIA-TAONVIDIA-TAO
83

tao-train-reid

Train, evaluate, export, and run inference for TAO person re-identification models.

Official
Advanced
NVIDIA-TAONVIDIA-TAO
83

tao-train-mask2former

Automates Mask2Former model training, evaluation, export, quantization, and inference.

Official
Advanced