paidf-auto-labeling

Plan, run, and debug PAIDF auto-labeling pipelines that convert video and image datasets into annotation artifacts.

3.2k|370|Updated Feb 25, 2026
One-click install
npx skills add https://github.com/NVIDIA/skills --skill paidf-auto-labeling
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: paidf-auto-labeling
Source: https://github.com/NVIDIA/skills/tree/main/skills/paidf-auto-labeling
Command: npx skills add https://github.com/NVIDIA/skills --skill paidf-auto-labeling

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Getting started with NVIDIA's Physical AI Data Factory (PAIDF) Auto-Labeling requires coordinating many moving parts: scenario planning, cookbook authoring, prompt and question-bank writing, stage configuration, and workflow-runner execution. This Skill routes each request to the correct specialized reference so agents follow the validated workflow instead of inventing one.

Core Features & Use Cases

  • Guided onboarding: Confirms critical inputs (data path, output path, VLM/LLM endpoints, model cache, GPU ids) before any run, then proves the stack with a shipped example cookbook before customization.
  • Task routing: Directs requests to specialized references for scenario planning, cookbook authoring, prompt authoring, pipeline migration, video data augmentation, event/person attribute search, event-verification reasoning, runner debugging, and per-stage configuration.
  • Safety guardrails: Requires container dry-runs before real execution, forbids committing secrets or absolute paths, and mandates raising max_tokens for reasoning-capable models.
  • Use Case: A user with warehouse-safety videos asks how to produce auto-labels; the Skill sequences plan → adapt cookbook → author prompts → configure detection-and-tracking → dry-run → execute and validate.

Quick Start

Ask your agent: "I cloned the PAIDF repo and have my own video data — help me get started producing auto-labels for my domain."

Frequently Asked Questions about paidf-auto-labeling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I get started with PAIDF auto-labeling on my own video data?

Confirm critical inputs first: data path, output path, VLM/LLM endpoints and model names, model cache, and GPU ids. Then run a shipped example cookbook with --container-dry-run to prove the stack, plan your scenario, adapt the closest cookbook, author prompts, and dry-run before executing.

How do I run the video data augmentation pipeline?

Use the shipped cookbook at cookbooks/video_data_augmentation/configs/pipeline_video.yaml through workflow-runner:main. Always run with --container-dry-run first, pass pre-flight checks for media, SAM3 weights, endpoints, and GPUs, then obtain explicit approval before the real Docker run.

Should I use RF-DETR or SAM3 for detection and tracking?

Choose RF-DETR when COCO classes and stable track IDs are sufficient, such as traffic or person inventory. Choose SAM3 when target objects are not COCO classes, when masks or contours are required, or when a VLM should reason over an annotated ID video.

Why are my visual_qa or reasoning outputs empty or truncated?

Reasoning-capable models such as Gemini 3 Flash spend part of the token budget on internal thinking, so a low max_tokens cap truncates outputs. Raise max_tokens to a generous value like 32768 on the visual_qa and reasoning LLM substages for reasoning models.

Can I commit API keys or endpoint URLs in a PAIDF cookbook?

No. Committed cookbooks must use placeholder paths and bare environment variable names like NVIDIA_API_KEY, which forwards the host value into containers at runtime. Never write NAME=value secrets into YAML, logs, or dry-run output.

When should I migrate an existing annotation repo instead of adapting a cookbook?

Use the pipeline-migration path when adopting an external annotation or dataset-generation repository into PAIDF. It classifies each source stage as cookbook-only reuse, an extension of an existing service, or a new generic capability service with a stable sidecar contract.