skills.rest
silverstein
silversteinCOMMUNITY
Explore all skills by this author →
Website𝕏@matsilversteinGitHub

pufferlib

Create vectorized PufferEnv environments and train PPO policies with PuffeRL.

21|2|Updated Dec 8, 2025
Search Tags:#reinforcement learning#vectorized environments#gymnasium#pettingzoo#pufferenv#ppo training#pytorch policy
One-click install
npx skills add https://github.com/silverstein/claude-scientific-skills-desktop --skill pufferlib-silverstein
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: pufferlib
Source: https://github.com/silverstein/claude-scientific-skills-desktop/tree/main/corpus/pufferlib
Command: npx skills add https://github.com/silverstein/claude-scientific-skills-desktop --skill pufferlib-silverstein

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pufferlib, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This skill helps you set up fast reinforcement learning pipelines so you can train agents efficiently instead of spending time on slow environment simulation and inefficient training scaffolding.

Core Features & Use Cases

  • High-performance PPO training (PuffeRL): Use vectorized environments and an optimized PPO+LSTM training flow to reach millions of steps per second.
  • Custom environment development (PufferEnv): Implement Gymnasium-/PettingZoo-style tasks using the PufferEnv API with observation/action space helpers and templates.
  • Vectorization & performance optimization: Scale parallel simulation using PufferLib’s worker/buffer patterns and tune throughput (num_envs, num_workers, envs_per_worker), including multi-agent setups.
  • Policy development patterns: Build PyTorch policies (MLP/CNN/LSTM) and integrate training-ready actor/critic heads, including recurrence-friendly design.
  • Framework integration: Emulate and integrate environments from Gymnasium and PettingZoo so you can train on existing benchmarks with minimal friction.

Quick Start

Use the pufferlib skill to build and run a PPO training workflow for an environment like Procgen Coinrun, then optimize throughput by tuning vectorization parameters and using an LSTM-capable policy when needed.

Frequently Asked Questions about pufferlib

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train PPO agents with high-throughput parallel simulation?▼

Vectorized environments allow parallel simulation by running multiple environment instances simultaneously. PufferLib uses worker and buffer patterns with configurable parameters like num_envs and num_workers to maximize throughput during PPO training.

Can I use PufferLib with existing Gymnasium and PettingZoo environments?▼

PufferLib supports integrating existing Gymnasium and PettingZoo environments. You can emulate these standard environments and train on them directly, minimizing setup friction when porting established benchmarks into the high-throughput pipeline.

How do I develop a custom reinforcement learning environment using PufferEnv?▼

You develop a custom PufferEnv environment by using the PufferEnv API alongside provided observation and action space helpers. This allows you to implement Gymnasium or PettingZoo-style tasks tailored for high-throughput vectorized simulation.

Does PufferLib support PyTorch policies with LSTM recurrence for PPO training?▼

PufferLib supports building PyTorch policies including MLP, CNN, and LSTM architectures. You can integrate training-ready actor and critic heads with recurrence-friendly designs to handle complex sequential decision-making during PPO training.

What is the best way to optimize throughput for multi-agent vectorized environments?▼

To optimize throughput for multi-agent vectorized environments, tune parameters like num_envs, num_workers, and envs_per_worker. PufferLib's optimized batching and worker/buffer patterns scale parallel simulation effectively for multi-agent setups.

Related Skills

Explore More Software Engineering Skills→
dralkhdralkh
74

pufferlib

Accelerate PufferLib reinforcement learning training with vectorized environments and PPO workflows.

Community
Advanced
robinbarvaagrobinbarvaag

pufferlib

Accelerate reinforcement learning training with vectorized environments and PPO.

Community
Advanced
jackspacejackspace
16

pufferlib

Optimize high-throughput PPO+LSTM reinforcement learning training with vectorized environments.

Community
Advanced
shushuznshushuzn
4

pufferlib

Run parallel environment simulation and PPO optimization for reinforcement learning training.

Community
Advanced