sglang-mixtral-quark-int4fp8-moe-optimization
CommunityOptimize Mixtral INT4/FP8 MoE for AMD hardware.
AuthorBBuf
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill provides a production-ready workflow for online INT4-FP8 MoE quantization of Mixtral MoE models on AMD ROCm hardware, enabling efficient deployment and testing.
Core Features & Use Cases
- Online quantization path for Mixtral-8x7B MoE checkpoints on ROCm/AMD.
- Quantizes high-precision MoE weights to packed INT4 and stores per-weight scales for FP8-style MoE execution.
- Guides auditing and extension of Mixtral AMD quantization in PR workflows, including GSM8K regression tests and diff history.
Quick Start
Start by configuring the AMD online int4fp8_moe quantization workflow for Mixtral-8x7B and run the associated GSM8K regression test.
Dependency Matrix
Required Modules
None requiredComponents
references
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: sglang-mixtral-quark-int4fp8-moe-optimization Download link: https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/archive/main.zip#sglang-mixtral-quark-int4fp8-moe-optimization Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.