paper_rl__qc

Implement action chunking for TD-based RL in offline-to-online continuous control.

2|Updated Mar 20, 2026
One-click install
npx skills add https://github.com/Gonglitian/agent-skills --skill paper-rl-qc
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: paper_rl__qc
Source: https://github.com/Gonglitian/agent-skills/tree/main/skills/papers/paper_rl__qc
Command: npx skills add https://github.com/Gonglitian/agent-skills --skill paper-rl-qc

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Action chunking enables TD-based RL to operate with temporally extended decisions.

Core Features & Use Cases

  • Action chunking supports long-horizon decision making in offline-to-online RL.
  • Two agent variants (ACFQL and ACRLPD) with n-step bootstrapping for stable learning.
  • Integrates with main.py and main_online.py for offline-to-online and online-only workflows.

Quick Start

Run offline-to-online experiments with a best-of-N or distill-DDPG actor on a long-horizon task.

Frequently Asked Questions about paper_rl__qc

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How does action chunking help with long-horizon reinforcement learning tasks?

Action chunking enables TD-based reinforcement learning to operate with temporally extended decisions. This approach stabilizes learning and improves performance on long-horizon, sparse-reward continuous control tasks.

How do I run offline-to-online RL fine-tuning using action chunking?

You run offline-to-online RL fine-tuning by executing the main.py and main_online.py workflows. These scripts support best-of-N or distill-DDPG actors for online fine-tuning from offline data on long-horizon tasks.

Can I use n-step returns with offline-to-online reinforcement learning?

Yes, the ACFQL and ACRLPD agent variants support n-step bootstrapping. This mechanism provides stable learning when combined with action chunking for offline-to-online reinforcement learning workflows.

Do I need to specify a horizon length for temporally extended decisions in continuous control?

Yes, you must define the horizon_length parameter to set the chunk size. This determines the temporal extension of decisions for TD-based reinforcement learning in continuous control environments.

What is the best way to handle sparse-reward manipulation tasks in offline-to-online RL?

Using action chunking with TD-based reinforcement learning is an effective approach for sparse-reward manipulation tasks. It leverages n-step returns and offline data for online fine-tuning, addressing the instability of long-horizon decision making.