Skills
.Work. You
Rest

jetson-speculative-decoding

Official

Speed up Jetson vLLM token generation

Software Engineering#benchmarking#speculative decoding#vllm#jetson#draft model#eagle-3
AuthorNVIDIA-AI-IOT
Version1.0.0
Installs0

System Documentation

What problem does it solve?

This Skill helps you reduce slow token generation on Jetson vLLM servers when time-per-output-token is the bottleneck, not initial response latency.

Core Features & Use Cases

  • EAGLE-3 Speculative Decoding: Enable a published EAGLE-3 head for supported Jetson deployments to improve decode speed.
  • Draft-Model Fallback: Use a small same-family draft model when no EAGLE-3 head is available.
  • Jetson-Specific Guardrails: Applies the right runtime path, memory headroom adjustments, and device-specific limits for Thor and AGX Orin systems.
  • Benchmark Validation: Verifies that speculation actually helps by comparing before-and-after benchmark results at low concurrency.

Quick Start

Ask the assistant to update your existing Jetson vLLM launch command with the correct speculative decoding configuration, then benchmark the server before and after the change to confirm it improves performance.

Dependency Matrix

Required Modules

None required

Components

Standard package

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: jetson-speculative-decoding
Download link: https://github.com/NVIDIA-AI-IOT/jetson-device-skills/archive/main.zip#jetson-speculative-decoding

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 537,000+ vetted skills library on demand.