stable-baselines3

Train reinforcement learning agents with Stable Baselines3 algorithms and Gymnasium environments.

Updated Oct 7, 2022
One-click install
npx skills add https://github.com/tamagusko/linux-cfg --skill stable-baselines3-tamagusko
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: stable-baselines3
Source: https://github.com/tamagusko/linux-cfg/tree/main/dotfiles/claude/skills/stable-baselines3
Command: npx skills add https://github.com/tamagusko/linux-cfg --skill stable-baselines3-tamagusko

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires stable-baselines3, gymnasium, numpy, and includes scripts (resource) and references (resource) components.

What problem does it solve? Training reinforcement learning agents involves many error-prone details: choosing the right algorithm, building valid Gymnasium environments, configuring vectorized training, and setting up evaluation and checkpointing. This Skill provides tested templates and reference guides that prevent common mistakes like invalid observation spaces, misconfigured callbacks, and incorrect VecEnv usage. ## Core Features & Use Cases - Algorithm Selection Guidance: Detailed comparison of PPO, SAC, TD3, DQN, A2C, DDPG, HER, and RecurrentPPO with hyperparameter recommendations and a decision tree for picking the right algorithm. - Ready-to-Use Templates: Scripts for training agents with evaluation and checkpoint callbacks, evaluating trained models with video recording, and building custom Gymnasium environments validated with SB3's env_checker. - Vectorized Training & Callbacks: In-depth references on DummyVecEnv vs SubprocVecEnv, VecNormalize, VecFrameStack, and custom callback patterns for early stopping, learning rate scheduling, and TensorBoard logging. - Use Case: A robotics researcher needs to train a SAC agent on a custom continuous-control environment. They adapt the custom environment template, validate it with check_env, then train with the training script using gradient_steps=-1 and EvalCallback to save the best model. ## Quick Start Train a PPO agent on CartPole-v1 using the training template with four parallel environments and evaluation callbacks.

Frequently Asked Questions about stable-baselines3

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a PPO agent with Stable Baselines3?

Create a Gymnasium environment, instantiate PPO with a policy like "MlpPolicy", and call model.learn with a total_timesteps budget. For faster training, wrap the environment with make_vec_env using several parallel environments before training.

Which Stable Baselines3 algorithm should I use for continuous control?

SAC is the recommended choice for continuous control because it is sample-efficient and stable with automatic entropy tuning. TD3 is a good alternative when you prefer a deterministic policy, while PPO works when simulations are cheap.

How do I create a custom Gymnasium environment for Stable Baselines3?

Subclass gymnasium.Env, define action_space and observation_space, and implement reset returning (observation, info) and step returning (observation, reward, terminated, truncated, info). Validate it with stable_baselines3.common.env_checker.check_env before training.

Does Stable Baselines3 support image observations?

Yes, image observations must be np.uint8 arrays in range [0, 255], preferably in channel-first format (channels, height, width). Use the "CnnPolicy" policy; SB3 normalizes images by dividing by 255 unless you set normalize_images=False.

Why is my Stable Baselines3 training slow or unstable?

Slow training often means you are not using vectorized environments; use SubprocVecEnv for compute-heavy environments. Instability usually comes from poorly tuned learning rates, unscaled rewards, or the wrong algorithm for your action space.

When should I not use Stable Baselines3 for reinforcement learning?

Avoid SB3 for high-performance massively parallel training, multi-agent systems, or custom vectorized environments, where a library like pufferlib is better suited. SB3 targets single-agent Gymnasium-based experiments and prototyping.