skills.rest
Supporter09
Supporter09COMMUNITY
Explore all skills by this author →
GitHub

speculative-decoding

Accelerate LLM token generation with speculative decoding strategies.

Updated May 4, 2026
Search Tags:#llm#inference#speculative decoding#medusa#latency optimization#lookahead decoding#parallel generation
One-click install
npx skills add https://github.com/Supporter09/Face_Anti_Spoofing_Biometric --skill speculative-decoding-supporter09
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: speculative-decoding
Source: https://github.com/Supporter09/Face_Anti_Spoofing_Biometric/tree/main/.claude/skills/speculative-decoding
Command: npx skills add https://github.com/Supporter09/Face_Anti_Spoofing_Biometric --skill speculative-decoding-supporter09

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps you reduce large language model latency and increase throughput when sequential token generation becomes the bottleneck.

Core Features & Use Cases

  • Draft-model speculative decoding: Use a smaller assistant model to propose tokens and let the target model verify them in parallel.
  • Medusa multiple decoding heads: Add parallel prediction heads to a model to generate and validate future tokens without a separate draft model.
  • Lookahead decoding: Apply Jacobi-style parallel generation and n-gram verification for plug-and-play speedups on existing models.
  • Use Case: A team serving a chatbot or code assistant can choose the fastest decoding strategy that fits its training budget, deployment constraints, and quality requirements.

Quick Start

Use the speculative-decoding skill to recommend the best acceleration method for a target LLM deployment and explain how to configure it safely.

Frequently Asked Questions about speculative-decoding

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reduce LLM inference latency without losing output quality?▼

To reduce LLM inference latency without quality loss, you can apply speculative decoding techniques like draft-model verification or Medusa heads to generate and validate tokens in parallel.

What's the best way to speed up token generation for a high-throughput chatbot?▼

The best way to speed up token generation for high-throughput workloads is using lookahead decoding, which applies Jacobi-style parallel generation and n-gram verification for plug-and-play speedups.

Does Medusa decoding require a separate draft model to accelerate generation?▼

Medusa decoding does not require a separate draft model. It adds parallel prediction heads directly to the target model to generate and validate future tokens independently.

Can I use speculative decoding for real-time code generation assistants?▼

You can use speculative decoding for real-time code generation assistants. It accelerates token generation by letting a smaller assistant model propose tokens for the target model to verify in parallel.

When should I choose lookahead decoding over draft-model speculative decoding?▼

Choose lookahead decoding over draft-model speculative decoding when you need a plug-and-play speedup on existing models without training or deploying a separate assistant model.

Why does sequential token generation bottleneck my LLM serving throughput?▼

Sequential token generation bottlenecks LLM serving throughput because the model generates one token at a time. Parallel verification methods like Medusa or draft-model decoding break this sequential barrier.

Related Skills

Explore More Software Engineering Skills→
uv-xiaouv-xiao
1

uv-speculative-decoding

Accelerate LLM inference with speculative decoding and Medusa multiple heads.

Community
Advanced
zechenzhangAGIzechenzhangAGI
11.5k

speculative-decoding

Reduce LLM inference latency using draft models, Medusa heads, and Lookahead Decoding.

Community
Intermediate
ovachieverovachiever
52

serving-llms-vllm

Deploy OpenAI-compatible LLM APIs with vLLM for high-throughput inference.

Community
Advanced
doanchienthangdevdoanchienthangdev
4

efficient-ai

Compress PyTorch models using quantization, pruning, and knowledge distillation.

Community
Advanced