fine-tuning-serving-openpi

Fine-tune and serve OpenPI pi0 robot policies using JAX or PyTorch backends.

2|Updated Aug 15, 2026
One-click install
npx skills add https://github.com/Jensen-Yao/agents-skills --skill fine-tuning-serving-openpi-jensen-yao
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: fine-tuning-serving-openpi
Source: https://github.com/Jensen-Yao/agents-skills/tree/main/skills/openpi
Command: npx skills add https://github.com/Jensen-Yao/agents-skills --skill fine-tuning-serving-openpi-jensen-yao

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires uv, jax, torch, transformers, and includes references (resource) components.

What problem does it solve? Adapting Physical Intelligence's OpenPI vision-language-action models (pi0, pi0-fast, pi0.5) to custom robot datasets and deploying them as low-latency inference servers involves complex setup, checkpoint conversion, and debugging that this Skill streamlines into repeatable workflows. ## Core Features & Use Cases - JAX and PyTorch Fine-Tuning: Config-driven training on LeRobot or RLDS datasets with normalization statistics computation and multi-GPU FSDP/DDP support. - Checkpoint Conversion: Convert JAX checkpoints to PyTorch format with required transformers patches for deployment-friendly inference. - Policy Serving: Launch WebSocket policy servers for ALOHA, DROID, and LIBERO environments with remote client integration patterns. - Use Case: A robotics engineer fine-tunes pi0.5 on a custom DROID dataset, computes norm stats, trains on an A100 cluster, then serves the checkpoint so a robot client can request action chunks over WebSocket. ## Quick Start Ask the agent to fine-tune an OpenPI pi0.5 model on your LeRobot dataset and serve the resulting checkpoint as a policy inference server.

Frequently Asked Questions about fine-tuning-serving-openpi

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune OpenPI pi0.5 on a custom dataset?▼

Copy the closest config in src/openpi/training/config.py, set your LeRobot repo_id, run compute_norm_stats.py, then launch training with scripts/train.py. Finally serve the checkpoint with serve_policy.py to validate inference.

How to convert OpenPI JAX checkpoints to PyTorch?▼

Run examples/convert_jax_model_to_pytorch.py with the checkpoint directory, config name, and output path. Apply the transformers_replace patches to the installed transformers package first, or training and inference will diverge.

What GPU memory is needed for OpenPI fine-tuning?▼

Serving pi0.5 inference needs about 24 GB VRAM, while JAX fine-tuning of pi0.5 requires roughly 60 GB on an A100 80GB. Use fsdp_devices to shard across multiple GPUs when memory is limited.

Why does OpenPI training fail with missing norm stats?▼

Every config requires precomputed normalization statistics before training. Run uv run scripts/compute_norm_stats.py --config-name <name> whenever the config, dataset, or transforms change.

Can OpenPI policies run on a remote robot client?▼

Yes, serve_policy.py exposes a WebSocket API on port 8000. Install the lightweight openpi-client package on the robot and call WebsocketClientPolicy.infer with observations containing image, state, and prompt keys.

When should I use OpenVLA-OFT instead of OpenPI?▼

Use OpenVLA-OFT when fine-tuning OpenVLA with continuous action heads and LoRA, or when reproducing OpenVLA-OFT paper results on LIBERO or ALOHA. OpenPI targets the pi0 model family with flow-matching or autoregressive architectures.