What problem does it solve?
PufferLib eliminates slow, single-threaded reinforcement learning workflows by providing highly optimized parallel environment simulation and training so researchers and engineers can iterate faster and scale experiments without rewriting environment code.
Core Features & Use Cases
- High-throughput training: PuffeRL implements an optimized PPO+LSTM trainer designed for millions of steps per second on GPU-backed training.
- Vectorized environments: Zero-copy shared-memory buffers, busy-wait worker flags, and hierarchical vectorization for serial, multiprocessing, and async modes.
- Multi-agent & integration: Native multi-agent support and seamless integration with Gymnasium, PettingZoo, Atari, Procgen, NetHack, and custom PufferEnv implementations.
- Developer tooling: Templates and reference guides for environment development, policy architectures (CNN, LSTM, attention), distributed torchrun training, logger integrations, and performance profiling.
Quick Start
Use pufferlib to train a PPO agent on procgen-coinrun with 256 vectorized environments on GPU.