eagle3-new-model

Configure EAGLE3 offline pipeline YAML files for new model checkpoints.

3.4k|535|Updated Apr 23, 2024
One-click install
npx skills add https://github.com/NVIDIA/Model-Optimizer --skill eagle3-new-model
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: eagle3-new-model
Source: https://github.com/NVIDIA/Model-Optimizer/tree/main/.agents/skills/eagle3-new-model
Command: npx skills add https://github.com/NVIDIA/Model-Optimizer --skill eagle3-new-model

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps users configure new EAGLE3 model checkpoints for offline pipeline execution without manually designing launcher YAML files or selecting incompatible settings.

Core Features & Use Cases

  • Launcher Configuration Guidance: Creates a new hf_offline_eagle3.yaml configuration by adapting the closest existing model example.
  • Backend Selection Support: Determines whether to use vLLM, HF, or TRT-LLM hidden-state dumping based on model characteristics such as multimodal support, attention patterns, and deployment compatibility.
  • Model-Specific Adjustments: Handles configuration details including trust-remote-code requirements, MoE expert dimensions, and tokenizer environment settings for new checkpoints.

Quick Start

Ask the skill to configure an EAGLE3 offline pipeline YAML for a new model checkpoint by selecting the correct backend and adapting an existing example.

Frequently Asked Questions about eagle3-new-model

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I configure a YAML file for a new EAGLE3 model checkpoint?

To configure a new EAGLE3 model checkpoint, you adapt the closest existing hf_offline_eagle3.yaml example. This process adjusts model-specific parameters like hidden-state dump backend selection and GPU settings to prepare optimized training and benchmark workflows.

How does backend selection work for EAGLE3 hidden-state dumping?

Backend selection for hidden-state dumping chooses between vLLM, HF, or TRT-LLM based on model characteristics. It evaluates multimodal support, attention patterns, and deployment compatibility to determine the optimal backend for the pipeline.

Can I use vLLM or TRT-LLM for multimodal model onboarding in EAGLE3?

Yes, EAGLE3 onboarding supports multimodal models by selecting the appropriate backend. The configuration process checks model characteristics to ensure compatibility, adjusting settings like trust-remote-code requirements and tokenizer environments for new checkpoints.

What configurations are needed for MoE expert dimensions in EAGLE3?

Configuring MoE expert dimensions requires adjusting model-specific parameters in the offline pipeline YAML. The skill handles these adjustments automatically by adapting reference configurations to fit the new checkpoint's architecture.

What is the best way to set up GPU configurations for EAGLE3 offline pipelines?

The best way to set up GPU configurations is by adapting an existing reference YAML file. This approach ensures launcher setup and GPU adjustments match the model's specific requirements without manually designing incompatible settings.

Why does my EAGLE3 model configuration require trust-remote-code settings?

Trust-remote-code settings are required when onboarding new checkpoints that use custom model architectures. The configuration process identifies this necessity and applies the correct environment parameters to ensure the offline pipeline executes properly.