fine-tuning-serving-openpi

Fine-tune and serve OpenPI pi0 robot policies using JAX or PyTorch backends.

13.0k|930|Updated Nov 3, 2025
One-click install
npx skills add https://github.com/Orchestra-Research/AI-research-SKILLs --skill fine-tuning-serving-openpi-orchestra-research
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: fine-tuning-serving-openpi
Source: https://github.com/Orchestra-Research/AI-research-SKILLs/tree/main/18-multimodal/openpi
Command: npx skills add https://github.com/Orchestra-Research/AI-research-SKILLs --skill fine-tuning-serving-openpi-orchestra-research

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires uv, jax, torch, transformers, and includes references (resource) components.

What problem does it solve? Adapting Physical Intelligence's OpenPI vision-language-action models (pi0, pi0-fast, pi0.5) to custom robot datasets involves complex setup across JAX and PyTorch backends, checkpoint conversion, normalization statistics, and low-latency policy serving, all of which are error-prone without a guided workflow. ## Core Features & Use Cases - JAX and PyTorch Fine-Tuning: Config-driven training on LeRobot or RLDS datasets with norm stats computation, FSDP sharding, and multi-GPU DDP support. - Checkpoint Conversion: Convert JAX checkpoints to PyTorch format with required transformers patches for deployment-friendly inference. - WebSocket Policy Serving: Run inference servers for ALOHA, DROID, and LIBERO environments with a lightweight remote client for robot integration. - Use Case: A robotics researcher fine-tunes pi0.5 on a custom DROID dataset, computes normalization statistics, trains on an A100 cluster, then serves the checkpoint over WebSocket so a remote robot client can request action chunks in real time. ## Quick Start Ask the AI to clone the OpenPI repository, install the workspace with uv, and start a pretrained DROID policy server, then connect a WebSocket client to run inference.

Frequently Asked Questions about fine-tuning-serving-openpi

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune OpenPI pi0 models on a custom dataset?▼

Copy the closest config in src/openpi/training/config.py, set your LeRobot repo_id and matching weight loader, then run compute_norm_stats.py before launching train.py with your config name. Always recompute norm stats after any dataset or transform change.

How to convert OpenPI JAX checkpoints to PyTorch?▼

Run examples/convert_jax_model_to_pytorch.py with the JAX checkpoint directory, config name, and output path. First apply the required transformers patches from src/openpi/models_pytorch/transformers_replace into your installed transformers package.

What GPU is needed to serve or fine-tune pi0.5?▼

Serving pi0.5 inference needs one GPU with about 24 GB VRAM, while JAX fine-tuning requires an A100 80GB using roughly 60 GB. PyTorch DDP training uses about 40 GB per GPU across 1-8 GPUs.

Why does OpenPI training fail with missing norm stats?▼

Every OpenPI config requires precomputed normalization statistics before training starts. Run uv run scripts/compute_norm_stats.py with your config name, and re-run it whenever the config, dataset, or transforms change.

OpenPI vs OpenVLA-OFT for robot policy fine-tuning?▼

OpenPI covers pi0, pi0-fast, and pi0.5 flow-matching VLA models with JAX and PyTorch backends for ALOHA, DROID, and LIBERO. Use OpenVLA-OFT instead when you need continuous action heads with LoRA or must reproduce OpenVLA-OFT paper results.

Why does the OpenPI policy server exit with code 137?▼

Exit code 137 indicates an out-of-memory kill. Set XLA_PYTHON_CLIENT_PREALLOCATE=false and XLA_PYTHON_CLIENT_ALLOCATOR=platform, or reduce XLA_PYTHON_CLIENT_MEM_FRACTION to limit JAX memory preallocation.