skills.rest
tamagusko
tamaguskoCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

pufferlib

Train reinforcement learning agents with vectorized environments and optimized PPO implementation.

Updated Oct 7, 2022
Search Tags:#multi-agent#reinforcement learning#pytorch#vectorization#ppo#pufferlib#gymnasium
One-click install
npx skills add https://github.com/tamagusko/linux-cfg --skill pufferlib-tamagusko
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: pufferlib
Source: https://github.com/tamagusko/linux-cfg/tree/main/dotfiles/claude/skills/pufferlib
Command: npx skills add https://github.com/tamagusko/linux-cfg --skill pufferlib-tamagusko

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pufferlib, torch, numpy, gymnasium, pettingzoo, and includes scripts (resource) and references (resource) components.

What problem does it solve? Standard reinforcement learning training is slow, often bottlenecked by environment simulation and inefficient vectorization. This Skill helps you build and train RL agents at millions of steps per second using PufferLib's optimized PPO implementation (PuffeRL), native multi-agent support, and high-performance vectorization. ## Core Features & Use Cases - High-Performance Training: Train agents with PuffeRL achieving 1M-4M steps per second, with CLI or Python APIs, distributed multi-GPU support, and WandB/Neptune logging. - Custom Environment Development: Create single-agent or multi-agent environments with the PufferEnv API using provided templates, then optimize from Python to C for 100M+ steps per second. - Framework Integration: Wrap environments from Gymnasium, PettingZoo, Atari, Procgen, NetHack, Minigrid, and more, or use the 20+ pre-built Ocean suite environments. - Use Case: Train a PPO agent on Procgen CoinRun across 256 parallel environments on a CUDA GPU, monitor progress in Weights & Biases, and checkpoint the model every 100 iterations. ## Quick Start Use the pufferlib skill to train a PPO agent on the procgen-coinrun environment with 256 parallel environments on CUDA.

Frequently Asked Questions about pufferlib

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a PPO agent with PufferLib?▼

Create a vectorized environment with pufferlib.make(), define a PyTorch policy, and pass both to the PuffeRL trainer. Then loop over trainer.evaluate(), trainer.train(), and trainer.mean_and_log() for each iteration, or use the puffer train CLI command.

PufferLib vs stable-baselines3 for reinforcement learning?▼

PufferLib targets maximum throughput, achieving 2-10x speedups through optimized vectorization and its PuffeRL PPO implementation, with native multi-agent support. Stable-baselines3 is better suited for quick prototyping and standard algorithms with extensive documentation.

Does PufferLib support multi-agent reinforcement learning?▼

Yes, PufferLib has native multi-agent support and treats single-agent and multi-agent environments uniformly. It integrates directly with PettingZoo parallel and AEC environments, and supports shared or independent policy parameters across agents.

Can I use Gymnasium environments with PufferLib?▼

Yes, wrap any Gymnasium environment with pufferlib.emulate() or use pufferlib.make('gym-EnvName'). PufferLib automatically flattens complex observation and action spaces for efficient vectorization while maintaining compatibility.

Why is my PufferLib training throughput low?▼

Low throughput usually comes from too few parallel environments, unoptimized Python environment code, or an underutilized GPU. Increase num_envs and batch_size, enable torch.compile, use in-place numpy operations, and consider C implementations for critical environment paths.

How do I create a custom environment for PufferLib?▼

Subclass PufferEnv, define observation and action spaces with make_space or make_discrete, and implement reset() and step() methods. Start from the env_template.py script, test locally, then vectorize with pufferlib.emulate() or make().

Related Skills

Explore More Software Engineering Skills→
dongzhuoyaodongzhuoyao

experiment-env-installer

Install reproducible Python environments for ML experiments with uv across compute backends.

Community
Advanced
NVIDIANVIDIA
3.2k

i4h-workflow-validate

Validate Isaac for Healthcare policies by running rollout episodes and recording verification HDF5 data.

Official
Advanced
DifanaDAPDifanaDAP

weights-and-biases

Automate ML experiment tracking and hyperparameter sweeps via the Python API.

Community
Advanced
biomaps-infrabiomaps-infra

bio-stable-baselines3

Implement and train reinforcement learning agents with Stable Baselines3 in Gymnasium environments.

Official
Advanced