pufferlib

Accelerate reinforcement learning training with parallel, zero-copy vectorized environments.

2|Updated Apr 12, 2026
One-click install
npx skills add https://github.com/Clay-HHK/claude-config --skill pufferlib-clay-hhk
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: pufferlib
Source: https://github.com/Clay-HHK/claude-config/tree/main/skills/pufferlib
Command: npx skills add https://github.com/Clay-HHK/claude-config --skill pufferlib-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires numpy, torch, and includes scripts (resource) and references (resource) components.

What problem does it solve?

PufferLib eliminates slow, single-threaded reinforcement learning workflows by providing highly optimized parallel environment simulation and training so researchers and engineers can iterate faster and scale experiments without rewriting environment code.

Core Features & Use Cases

  • High-throughput training: PuffeRL implements an optimized PPO+LSTM trainer designed for millions of steps per second on GPU-backed training.
  • Vectorized environments: Zero-copy shared-memory buffers, busy-wait worker flags, and hierarchical vectorization for serial, multiprocessing, and async modes.
  • Multi-agent & integration: Native multi-agent support and seamless integration with Gymnasium, PettingZoo, Atari, Procgen, NetHack, and custom PufferEnv implementations.
  • Developer tooling: Templates and reference guides for environment development, policy architectures (CNN, LSTM, attention), distributed torchrun training, logger integrations, and performance profiling.

Quick Start

Use pufferlib to train a PPO agent on procgen-coinrun with 256 vectorized environments on GPU.

Frequently Asked Questions about pufferlib

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I accelerate reinforcement learning training to millions of steps per second?

Accelerate reinforcement learning training by using optimized PPO+LSTM trainers and zero-copy shared-memory vectorized environments, enabling millions of steps per second on GPU-backed training.

Can I use PyTorch for multi-agent PPO training with PettingZoo environments?

Yes, you can use PyTorch for multi-agent PPO training with PettingZoo environments, as native multi-agent support and seamless integration with PettingZoo are provided.

What is the best way to vectorize Gymnasium environments for high-throughput RL?

Vectorize Gymnasium environments for high-throughput RL by using zero-copy shared-memory buffers, busy-wait worker flags, and hierarchical vectorization across serial, multiprocessing, and async modes.

Does distributed multi-GPU training work with PPO and LSTM policies?

Distributed multi-GPU training works with PPO and LSTM policies through integrated torchrun support, allowing you to scale experiments without rewriting environment code.

Do I need custom environment templates to build a PufferEnv?

You do not need to build from scratch, as templates and reference guides for environment development, policy architectures, and custom PufferEnv implementations are provided.

Why is single-threaded reinforcement learning slow and how does vectorization fix it?

Single-threaded reinforcement learning is slow due to bottlenecks, which vectorization fixes by implementing parallel environment simulation with shared-memory zero-copy buffers and busy-wait workers.