paper_rl__qc

Implement action chunking for TD-based RL in offline-to-online continuous control.

2|Updated Mar 20, 2026

Search Tags:#reinforcement-learning #rl #action-chunking #offline-to-online #n-step-returns #jax-flax

Ready to run

npx skills add https://github.com/Gonglitian/agent-skills --skill paper-rl-qc

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: paper_rl__qc
Source: https://github.com/Gonglitian/agent-skills/tree/main/skills/papers/paper_rl__qc
Command: npx skills add https://github.com/Gonglitian/agent-skills --skill paper-rl-qc

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Action chunking enables TD-based RL to operate with temporally extended decisions.

Core Features & Use Cases

Action chunking supports long-horizon decision making in offline-to-online RL.
Two agent variants (ACFQL and ACRLPD) with n-step bootstrapping for stable learning.
Integrates with main.py and main_online.py for offline-to-online and online-only workflows.

Quick Start

Run offline-to-online experiments with a best-of-N or distill-DDPG actor on a long-horizon task.

Frequently Asked Questions about paper_rl__qc

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How does action chunking help with long-horizon reinforcement learning tasks?▼

Action chunking enables TD-based reinforcement learning to operate with temporally extended decisions. This approach stabilizes learning and improves performance on long-horizon, sparse-reward continuous control tasks.

How do I run offline-to-online RL fine-tuning using action chunking?▼

You run offline-to-online RL fine-tuning by executing the main.py and main_online.py workflows. These scripts support best-of-N or distill-DDPG actors for online fine-tuning from offline data on long-horizon tasks.

Can I use n-step returns with offline-to-online reinforcement learning?▼

Yes, the ACFQL and ACRLPD agent variants support n-step bootstrapping. This mechanism provides stable learning when combined with action chunking for offline-to-online reinforcement learning workflows.

Do I need to specify a horizon length for temporally extended decisions in continuous control?▼

Yes, you must define the horizon_length parameter to set the chunk size. This determines the temporal extension of decisions for TD-based reinforcement learning in continuous control environments.

What is the best way to handle sparse-reward manipulation tasks in offline-to-online RL?▼

Using action chunking with TD-based reinforcement learning is an effective approach for sparse-reward manipulation tasks. It leverages n-step returns and offline data for online fine-tuning, addressing the instability of long-horizon decision making.