isaac-sim-remote

Execute Python remotely inside a running Isaac Sim via TCP socket to control stages, prims, cameras, and simulations.

4.0k|530|Updated May 28, 2025
One-click install
npx skills add https://github.com/isaac-sim/IsaacSim --skill isaac-sim-remote
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: isaac-sim-remote
Source: https://github.com/isaac-sim/IsaacSim/tree/main/.claude/skills/isaac-sim-remote
Command: npx skills add https://github.com/isaac-sim/IsaacSim --skill isaac-sim-remote

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Interacting with a running Isaac Sim instance normally requires manual UI work or embedding code inside the app. This Skill lets you send Python code over the isaacsim.code_editor.python_server TCP socket (port 8226) to automate stage creation, simulation control, screenshots, and scene inspection without touching the UI.

Core Features & Use Cases

  • Remote Code Execution: Send inline Python or script files to a running Isaac Sim, with named execution contexts, fire-and-forget tasks, per-request timeouts, and JSON-envelope argument injection.
  • Stage & Scene Automation: Create/open/save USD stages, inspect and modify prims, set transforms, control the viewport camera, and step or play the physics simulation.
  • Capture & Debugging: Take viewport or full-app screenshots, capture annotator data (depth, normals, segmentation), read console logs, and run health checks against the server.
  • Use Case: Launch Isaac Sim headless with --no-window, create a new stage with a DomeLight and Cube, warm up the renderer, and capture a viewport screenshot — all from shell commands via scripts/isaacsim_send.py.

Quick Start

Launch Isaac Sim with the python_server extension enabled, then use the isaac-sim-remote skill to run a health check and capture a viewport screenshot of the current stage.

Frequently Asked Questions about isaac-sim-remote

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I execute Python code in a running Isaac Sim remotely?

Enable the isaacsim.code_editor.python_server extension at launch with --enable isaacsim.code_editor.python_server, then send code via scripts/isaacsim_send.py over TCP port 8226. It supports inline code, script files, async top-level await, and returns JSON responses.

How do I take a screenshot in Isaac Sim headless mode?

Viewport screenshots work fully headless using replicator annotators via the viewport_screenshot or capture_annotator scripts. Full-app swapchain screenshots require a DISPLAY environment variable, such as DISPLAY=:99 with Xvfb, even in --no-window mode.

Why is the Isaac Sim python server port 8226 not open?

The extension is not enabled by default and must be passed with --enable isaacsim.code_editor.python_server at launch. The settings syntax --/exts/isaacsim.code_editor.python_server/enabled=true does not work. Wait for 'app ready' in the log before connecting.

Does Isaac Sim remote execution support async code?

Yes, the server detects top-level await and awaits the code automatically. Prefer await app_utils.update_app_async() over the sync update_app() inside async code to avoid asyncio reentrancy errors like 'Cannot enter into task'.

Why does set_world_pose fail on prims during simulation in Isaac Sim?

xform.set_world_pose() raises NotImplementedError, and XformPrim.set_world_poses can fail during simulation because warp arrays replace numpy arrays at runtime. Use raw USD xform ops via UsdGeom.Xformable to set translations while the simulation is playing.