skills.rest
BBuf
BBufCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

sglang-mixtral-quark-int4fp8-moe-optimization

Quantize Mixtral MoE models to INT4FP8 with per-expert weight packing on AMD ROCm.

721|65|Updated Apr 1, 2026
Search Tags:#quantization#amd#fp8#rocm#moe#int4#mixtral
One-click install
npx skills add https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS --skill sglang-mixtral-quark-int4fp8-moe-optimization
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sglang-mixtral-quark-int4fp8-moe-optimization
Source: https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/tree/main/skills/model-optimization/sglang/sglang-mixtral-quark-int4fp8-moe-optimization
Command: npx skills add https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS --skill sglang-mixtral-quark-int4fp8-moe-optimization

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill provides a production-ready workflow for online INT4-FP8 MoE quantization of Mixtral MoE models on AMD ROCm hardware, enabling efficient deployment and testing.

Core Features & Use Cases

  • Online quantization path for Mixtral-8x7B MoE checkpoints on ROCm/AMD.
  • Quantizes high-precision MoE weights to packed INT4 and stores per-weight scales for FP8-style MoE execution.
  • Guides auditing and extension of Mixtral AMD quantization in PR workflows, including GSM8K regression tests and diff history.

Quick Start

Start by configuring the AMD online int4fp8_moe quantization workflow for Mixtral-8x7B and run the associated GSM8K regression test.

Frequently Asked Questions about sglang-mixtral-quark-int4fp8-moe-optimization

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I quantize Mixtral MoE weights to INT4 and FP8 on AMD ROCm hardware?▼

Yes, the workflow applies online INT4FP8 quantization specifically to Mixtral-8x7B and related MoE checkpoints on AMD ROCm hardware, executing per-expert weight packing during runtime.

Does the Mixtral AMD quantization workflow support GSM8K regression testing?▼

Yes, the workflow guides auditing and extension of Mixtral AMD quantization in PR workflows and includes running associated GSM8K regression tests to track diff history and ensure model performance.

What is needed to run online weight quantization for Mixtral MoE on AMD?▼

Running online weight quantization for Mixtral MoE on AMD requires ROCm hardware setup and utilizes AMD-only quantization paths to enable per-expert weight packing and FP8-style scales in fused-MoE execution pipelines.

Can I track diff audits for Mixtral AMD quantization changes?▼

Yes, the workflow satisfies requirements for diff-audit tracking by guiding the auditing and extension of Mixtral AMD quantization directly within PR workflows, ensuring quantization changes are properly reviewed.

Related Skills

Explore More Software Engineering Skills→
amdamd
154

quark-onnx-ptq-workflow

Guide end-to-end post-training quantization of ONNX models with AMD Quark.

Official
Advanced
BBufBBuf
721

vllm-mixtral-quark-int4fp8-moe-optimization

Audit PR diffs for Mixtral Quark INT4-FP8 MoE optimization in vLLM.

Community
Advanced
hung-phanhung-phan
1

nemo

Scale LLM and MoE training across NVIDIA GPUs with 5D parallelism.

Community
Advanced
Orchestra-ResearchOrchestra-Research
11.5k

miles-rl-training

Train large-scale MoE models with FP8 and INT4 low-precision RL.

Official
Advanced