skills.rest
ricable
ricableCOMMUNITY
Explore all skills by this author →
GitHub

speculative-decoding

Reduce LLM decoding latency with speculative decoding techniques.

Updated Feb 15, 2026
Search Tags:#token generation#speculative decoding#medusa#latency optimization#llm inference#lookahead decoding
One-click install
npx skills add https://github.com/ricable/mcai --skill speculative-decoding-ricable
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: speculative-decoding
Source: https://github.com/ricable/mcai/tree/main/.agents/skills/speculative-decoding
Command: npx skills add https://github.com/ricable/mcai --skill speculative-decoding-ricable

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps you reduce LLM generation latency and increase throughput when serving large models for chat, code, or other real-time text generation workloads.

Core Features & Use Cases

  • Draft-model speculative decoding for faster token generation while preserving target-model quality.
  • Medusa multiple-head inference for single-model acceleration with minimal extra training.
  • Lookahead decoding for plug-and-play speedups without adding a separate draft model.
  • Use case: accelerate an API endpoint that must answer user prompts quickly under limited GPU capacity.

Quick Start

Ask the skill to compare speculative decoding, Medusa, and lookahead decoding for your model size, latency target, and deployment constraints.

Frequently Asked Questions about speculative-decoding

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reduce LLM inference latency without losing output quality?▼

Speculative decoding reduces LLM inference latency without quality loss by using parallel token verification and KV-cache-aware generation to accelerate text output for real-time workloads.

What is the difference between Medusa heads and lookahead decoding for token generation?▼

Medusa uses multiple-head inference for single-model acceleration requiring minimal training, while lookahead decoding provides plug-and-play speedups without adding a separate draft model.

How do I speed up token generation for an LLM API under limited GPU capacity?▼

You can speed up token generation under limited GPU capacity by applying speculative decoding, Medusa, or lookahead decoding strategies to increase throughput and reduce decoding delays.

Do I need a separate draft model to accelerate LLM inference?▼

You do not need a separate draft model if you use Medusa multiple-head inference or lookahead decoding, which enable single-model acceleration and plug-and-play speedups respectively.

Does speculative decoding work with my existing KV-cache-aware generation setup?▼

Yes, speculative decoding requires and integrates with KV-cache-aware generation and parallel token verification to maintain target-model quality while reducing decoding latency.

When should I not use draft models for LLM inference optimization?▼

You should avoid draft models when minimal extra training is unacceptable or deployment constraints prevent managing multiple models, opting instead for Medusa or lookahead decoding strategies.

Related Skills

Explore More Software Engineering Skills→
georgeguimaraesgeorgeguimaraes
11

context-optimization

Summarize and merge context segments to extend effective context windows.

Community
Advanced
zechenzhangAGIzechenzhangAGI
11.5k

speculative-decoding

Reduce LLM inference latency using draft models, Medusa heads, and Lookahead Decoding.

Community
Intermediate
Supporter09Supporter09

speculative-decoding

Accelerate LLM token generation with speculative decoding strategies.

Community
Advanced
handsomelong922handsomelong922

speculative-decoding

Accelerate LLM inference with speculative decoding, Medusa heads, and Lookahead Decoding.

Community
Advanced