pufferlib

Train reinforcement learning agents with parallel environments and PPO+LSTM algorithms.

Updated Dec 17, 2025

Search Tags:#reinforcement learning #ppo #gymnasium #lstm #pettingzoo #pufferenv

Ready to run

npx skills add https://github.com/robotlearning123/claude-scientific-skills --skill pufferlib-robotlearning123

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: pufferlib
Source: https://github.com/robotlearning123/claude-scientific-skills/tree/main/scientific-skills/pufferlib
Command: npx skills add https://github.com/robotlearning123/claude-scientific-skills --skill pufferlib-robotlearning123

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pufferlib, gymnasium, pettingzoo, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill unit simplifies the process of training reinforcement learning agents by providing a high-performance library for parallel environment simulation, custom environment development, and efficient training algorithms.

Core Features & Use Cases

High-Performance Training: Optimized PPO+LSTM training for millions of steps per second.
Custom Environment Development: Build custom environments using the PufferEnv API.
Vectorization and Performance: Achieve maximum throughput with optimized parallel simulation.
Policy Development: Build policies with CNNs, LSTMs, or custom architectures.
Environment Integration: Seamlessly integrate environments from Gymnasium, PettingZoo, and other frameworks.
Use Case: You can use this skill to train a reinforcement learning agent to play a game like "CartPole" or develop a custom environment for a specific task.

Quick Start

Start training a reinforcement learning agent using the PufferLib skill:

puffer train cartpole --train.device cuda --train.learning-rate 3e-4

Frequently Asked Questions about pufferlib

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I train reinforcement learning agents with high-performance parallel simulation?▼

Train reinforcement learning agents with parallel simulation by running optimized PPO+LSTM algorithms to achieve millions of steps per second. This approach maximizes throughput by utilizing vectorized environments for efficient training.

Can I use Gymnasium and PettingZoo environments for reinforcement learning training?▼

Yes, you can integrate Gymnasium and PettingZoo environments seamlessly for reinforcement learning training. The system supports these frameworks natively, allowing you to wrap existing environments for high-performance parallel simulation.

How do I build a custom reinforcement learning environment using the PufferEnv API?▼

Build a custom reinforcement learning environment by implementing the PufferEnv API to define your specific task logic. This allows you to create tailored simulations for specialized domains like gaming or robotics before training agents.

Does PufferLib support recurrent neural networks like LSTM for policy development?▼

Yes, PufferLib supports LSTM networks for policy development alongside CNNs and custom architectures. This allows you to build complex policies that handle temporal dependencies in reinforcement learning tasks during training.

What is the best way to accelerate reinforcement learning training for millions of steps?▼

Accelerate reinforcement learning training by utilizing high-performance parallel environment simulation and optimized PPO algorithms. This combination enables you to process millions of training steps per second efficiently on supported devices.

Do I need CUDA to train reinforcement learning agents with PufferLib?▼

Using CUDA is recommended to train reinforcement learning agents efficiently, as specified in the training command parameters. You can configure the training device to CUDA to accelerate the optimized PPO+LSTM computations.