skills.rest
isaac-sim
isaac-simOFFICIAL
Explore all skills by this author →
GitHub

data-collection-sim

Generates annotated synthetic training datasets using Isaac Sim Replicator writers and domain randomization.

4.0k|530|Updated May 28, 2025
Search Tags:#simulation#computer vision#isaac sim#dataset#domain randomization#synthetic data generation#replicator
One-click install
npx skills add https://github.com/isaac-sim/IsaacSim --skill data-collection-sim
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-collection-sim
Source: https://github.com/isaac-sim/IsaacSim/tree/main/skills/data-collection-sim
Command: npx skills add https://github.com/isaac-sim/IsaacSim --skill data-collection-sim

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires numpy, yaml, and includes scripts (resource) components.

What problem does it solve?

Manually collecting and labeling real-world training data for computer vision models is slow and expensive. This Skill builds headless synthetic data generation (SDG) pipelines in Isaac Sim 6 that produce annotated RGB, depth, segmentation, bounding box, and pose data at scale.

Core Features & Use Cases

  • Multiple Writers: Produce datasets via BasicWriter, KittiWriter, CocoWriter, CosmosWriter, PoseWriter, and DataVisualizationWriter from omni.replicator.core and isaacsim.replicator.writers.
  • Domain Randomization: Randomize object poses, camera positions, and lighting per frame using rep.functional APIs with seeded reproducibility.
  • Semantic Labeling: Tag prims with add_labels or rep.functional.modify.semantics so annotators emit correct segmentation and bounding boxes.
  • Use Case: Generate 1,000 annotated warehouse images with randomized forklift, pallet, and box placements to train an object detection model, then validate output frame counts and image quality with the included validation script.

Quick Start

Ask the AI to generate a synthetic warehouse dataset of 100 annotated frames using the warehouse_sdg.py script with a YAML config specifying objects, resolution, and output directory.

Frequently Asked Questions about data-collection-sim

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate synthetic training data with Isaac Sim Replicator?▼

Launch a headless SimulationApp, load a USD stage, tag prims with semantic labels, create a camera and render product, attach a writer like BasicWriter, then call rep.orchestrator.step in a capture loop. Randomize object and camera poses each frame for dataset diversity.

What writers are available in omni.replicator.core for dataset export?▼

omni.replicator.core provides BasicWriter for RGB, depth, segmentation, and bounding boxes, plus KittiWriter, CocoWriter, CosmosWriter, and FPSWriter. The isaacsim.replicator.writers module adds PoseWriter for 6-DoF pose estimation and DataVisualizationWriter for debug overlays.

How do I run Isaac Sim SDG scripts headless?▼

Run the script with $ISAAC_SIM_DIR/python.sh and set SimulationApp({"headless": True}) inside the script. Headlessness is controlled by the SimulationApp config, not a launcher flag, and isaac-sim.sh launches the full editor instead.

Why are my Replicator segmentation maps missing objects?▼

Annotators only emit labels for prims tagged with semantic data. Apply add_labels(prim, labels=[...], taxonomy="class") or rep.functional.modify.semantics to every prim you want annotated; both write the UsdSemantics LabelsAPI schema.

What is rt_subframes and how should I set it for SDG?▼

rt_subframes renders the same frame multiple times per capture to reduce ghosting and let materials converge. Use 4-8 for RTX Real-Time with DLSS Quality, and 16-32 for path tracing or scenes with heavy texture streaming.

When should I use mobility-gen instead of static-scene SDG?▼

Use mobility-gen for mobile-robot workflows that record trajectories and replay them with robot-mounted sensors in a two-phase process. Static-scene SDG with this Skill fits fixed environments where objects and cameras are randomized per frame.

Related Skills

Explore More Software Engineering Skills→
PubCyBerryPubCyBerry

semantic-labels

Assign USD SemanticsAPI labels and classes to scene prims for segmentation workflows.

Community
Intermediate
SharpAISharpAI
3.0k

dataset-annotation

Annotate datasets with bounding boxes, SAM2 segmentation, and DINOv3 visual grounding.

Official
Intermediate
mjunaidcamjunaidca
9

sensor-simulation

Configure simulated camera, LIDAR, IMU, and contact sensors in Gazebo for realistic perception data.

Community
Intermediate
elliewlh2094elliewlh2094

ros2-bag

Record and replay robot sensor data via CLI or Python and C++.

Community
Intermediate