skills.rest
inclusionAI
inclusionAIOFFICIAL
Explore all skills by this author →
WebsiteGitHub

add-archon-model

Guides adding new HuggingFace model architectures to the Archon training engine.

5.7k|588|Updated Feb 24, 2025
Search Tags:#archon#model integration#huggingface#moe#Tensor Parallelism#Reinforcement Learning#state dict adapter
One-click install
npx skills add https://github.com/inclusionAI/AReaL --skill add-archon-model
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: add-archon-model
Source: https://github.com/inclusionAI/AReaL/tree/main/.opencode/skills/add-archon-model
Command: npx skills add https://github.com/inclusionAI/AReaL --skill add-archon-model

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Adding support for a new HuggingFace model architecture to the Archon training engine requires implementing many interdependent components (model args, attention, FFN, RoPE, state dict adapters, parallelization, registration, and tests), and mistakes in any of them cause silent weight-mapping or parallelism errors.

Core Features & Use Cases

  • Architecture Analysis: Walks through reading a target model's HuggingFace config and modeling source to identify attention variants, FFN types, MoE support, RoPE variants, and normalization schemes.
  • Reference-Based Implementation: Selects the closest existing implementation (qwen2 for dense models, qwen3 for QK-norm/MoE models) and adapts each file: args.py, model.py, rope.py, state_dict_adapter.py, and parallelize.py.
  • Registration and Testing: Covers ModelSpec creation, auto-registration in the package init.py, and staged tests from CPU-only args tests to GPU forward-precision comparison against HuggingFace.
  • Use Case: A developer wants to train a Llama-family model with ArchonEngine; the skill guides them through mapping HF config fields, writing the state dict adapter, defining the TP/EP/CP/FSDP parallel plan, and verifying correctness with roundtrip tests.

Quick Start

Ask the assistant to add support for a specific HuggingFace model, such as "add Llama-3-8B support to ArchonEngine", and follow the guided steps.

Frequently Asked Questions about add-archon-model

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a new model to the Archon training engine?▼

Analyze the HuggingFace config and modeling source, pick the closest reference (qwen2 for dense models, qwen3 for QK-norm or MoE models), then implement args.py, model.py, rope.py, state_dict_adapter.py, and parallelize.py. Finally create a ModelSpec, register it in the package __init__.py, and add staged tests.

Which reference implementation should I copy when adding an Archon model?▼

Use qwen2 for dense-only models with standard GQA and no QK norm, since it is the simplest baseline. Use qwen3 when the target model has QK norm or MoE layers, since it supports QK norm, MoE routing, and shared experts.

Does Archon support MoE models like Mixtral or DeepSeek?▼

Yes, through the qwen3 reference implementation, which includes MoE modules, router gates, shared experts, and expert weight 3D-to-2D conversion in the state dict adapter. The parallelize function also applies expert parallelism for MoE layers.

Why do weights silently drop when loading a HuggingFace checkpoint into Archon?▼

This usually happens when the state dict adapter's from_hf_map is missing key mappings or mishandles keys set to None, such as rotary_emb.inv_freq. Verify with a roundtrip test asserting that archon-to-HF-to-archon conversion preserves all keys.

What tests should I write for a new Archon model implementation?▼

Follow the patterns in tests/experimental/archon: CPU-only args tests for from_hf_config, state dict roundtrip tests, meta-device weight completeness checks, and single-GPU forward-precision comparison against the HuggingFace reference when CUDA is available.

Related Skills

Explore More Software Engineering Skills→
DoanNgocCuongDoanNgocCuong

ray-train

Orchestrate distributed machine learning training across clusters with Ray Train.

Community
Advanced
DoanNgocCuongDoanNgocCuong

slime-rl-training

Run GRPO training jobs for Qwen3-4B with Megatron-LM and SGLang.

Community
Advanced
Metta-AIMetta-AI

cb.fabric-parity-gate

The requested URL returns an error. Please check your input and try again. Alternatively, you can try the same URL, wrapped in double quotes.

Official
Advanced
icyzhicyzh
1

huggingface-tokenizers

Train BPE, WordPiece, and Unigram tokenizers for HuggingFace Transformers.

Community
Advanced