skills.rest
tachyon-beep
tachyon-beepCOMMUNITY
Explore all skills by this author →
GitHub

offline-rl

Implement CQL, IQL, and BCQ for offline reinforcement learning from fixed datasets.

1|Updated Oct 29, 2025
Search Tags:#evaluation#CQL#reinforcement-learning#offline-rl#IQL#BCQ#data-efficiency
One-click install
npx skills add https://github.com/tachyon-beep/hamlet --skill offline-rl
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: offline-rl
Source: https://github.com/tachyon-beep/hamlet/tree/main/.claude/skills/yzmir-deep-rl/skills/offline-rl
Command: npx skills add https://github.com/tachyon-beep/hamlet --skill offline-rl

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill enables practitioners to perform offline reinforcement learning from fixed datasets without interacting with an environment, addressing issues like distribution shift and extrapolation errors by adopting conservative value estimation and robust evaluation strategies.

Core Features & Use Cases

  • Conservative offline RL methods: CQL, IQL, and BCQ to mitigate overestimation when data is limited or distribution shifts occur.
  • Offline training & evaluation: Train purely from logged data and evaluate using offline techniques to estimate policy performance without live rollout.
  • Real-world scenarios: Robotics from logged demonstrations, medical treatment policy learning from historical data, and recommendations improved from historical user interactions.

Quick Start

Prepare a fixed dataset, choose a method (CQL, IQL, or BCQ), train the model, and perform offline evaluation using RIS or model-based estimates.

Frequently Asked Questions about offline-rl

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train reinforcement learning models from fixed datasets without environment interaction?▼

Offline reinforcement learning trains policies entirely from fixed datasets without environment interaction. You apply conservative value estimation methods like CQL, IQL, or BCQ to mitigate distribution shift and extrapolation errors.

What is the best way to prevent extrapolation errors in offline reinforcement learning?▼

To prevent extrapolation errors in offline reinforcement learning, apply conservative value estimation techniques. Methods like CQL, IQL, and BCQ constrain the policy to avoid out-of-distribution actions when training from limited logged data.

Can I use offline RL for medical treatment policy learning from historical data?▼

Yes, offline RL suits medical treatment policy learning from historical data. It enables evaluating and training treatment policies using offline evaluation strategies like RIS without requiring live patient rollouts, prioritizing safety and data efficiency.

How do I evaluate policy performance in offline reinforcement learning without live rollout?▼

You evaluate policy performance in offline reinforcement learning using offline evaluation techniques. Apply RIS or model-based estimates directly to the fixed dataset to estimate policy performance safely without executing live environmental rollouts.

When should I choose IQL over CQL or BCQ for offline reinforcement learning?▼

Choose IQL when you need to avoid querying out-of-distribution actions entirely during offline reinforcement learning. CQL applies conservative value penalties, while BCQ restricts the action space, distinguishing their approaches to mitigating overestimation.

Why does offline reinforcement learning suffer from distribution shift?▼

Offline reinforcement learning suffers from distribution shift because the fixed training dataset does not cover the state-action space explored by the newly learned policy. This mismatch causes overestimation errors when evaluating unseen actions.

Related Skills

Explore More Education & Research Skills→
tadod12tadod12

llama-factory

Guides users to fine-tune LLaMA-Factory models via a WebUI no-code workflow.

Community
Advanced
ken-cavanagh-gleanken-cavanagh-glean
4

evaluation

Design evaluation frameworks with rubrics and LLM-as-judge workflows.

Community
Advanced
K-Dense-AIK-Dense-AI
33.0k

pyhealth

Train and evaluate clinical prediction models on EHR datasets.

Official
Advanced
ruvnetruvnet
4.4k

AgentDB Learning Plugins

Build and train reinforcement-learning plugins for autonomous agents via AgentDB.

Community
Intermediate