Verified:GitHub

speculative-decoding

Accelerate LLM inference with speculative decoding, Medusa heads, and Lookahead decoding.

1.0k|117|Updated Feb 26, 2026

Search Tags:#speculative decoding #medusa #inference speed #llm inference #lookahead decoding #parallel generation #tree attention

Ready to run

npx skills add https://github.com/OpenLAIR/dr-claw --skill speculative-decoding-openlair

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: speculative-decoding
Source: https://github.com/OpenLAIR/dr-claw/tree/main/skills/emerging-techniques/speculative-decoding
Command: npx skills add https://github.com/OpenLAIR/dr-claw --skill speculative-decoding-openlair

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires transformers, torch, and includes references (resource) components.

What problem does it solve?

Speculative decoding speeds up large language model inference by combining a small, fast draft model with a larger target model to generate and verify tokens in parallel, reducing latency without sacrificing quality.

Core Features & Use Cases

Speculative Decoding: Use a draft model to generate candidate tokens which the main model verifies in a single forward pass.
Medusa Heads: Attach multiple decoding heads to predict several future tokens in parallel, enabling tree-based candidate exploration.
Lookahead Decoding: Apply Jacobi-iteration-inspired lookahead to generate disjoint n-grams in parallel with a verification step.
Use Case: Deploy large models for real-time chat or code generation where latency is critical and compute budgets permit draft-head strategies.

Quick Start

Provide a prompt and choose Speculative Decoding, Medusa, or Lookahead to generate results faster.

Frequently Asked Questions about speculative-decoding

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I speed up LLM inference using speculative decoding?▼

Speculative decoding accelerates LLM inference by using a small draft model to generate candidate tokens, which the main target model then verifies in a single forward pass to reduce latency without losing quality.

What is Medusa multiple heads decoding for parallel token generation?▼

Medusa decoding attaches multiple decoding heads to a model to predict several future tokens in parallel, enabling tree-based candidate exploration for faster inference and increased throughput.

Can I use lookahead decoding with PyTorch and transformers to reduce latency?▼

Yes, lookahead decoding applies Jacobi-iteration-inspired parallel n-gram generation and verification using PyTorch and transformers to significantly lower inference latency for large model deployments.

Does speculative decoding require a separate draft model to work?▼

Speculative decoding requires a small fast draft model to generate candidate tokens, whereas Medusa heads and lookahead decoding achieve parallel generation natively without a separate draft model.

When should I use speculative decoding over standard LLM generation?▼

Use speculative decoding for real-time chat or code generation where latency is critical and compute budgets permit draft-head strategies, ensuring faster responses without sacrificing generation quality.

What is the difference between Medusa heads and lookahead decoding for LLM inference?▼

Medusa heads predict multiple future tokens in parallel using tree-based attention, while lookahead decoding uses Jacobi iterations to generate disjoint n-grams in parallel with a verification step.