sparse-quantization

What problem does it solve?

Sparse-quantization on MindSpore within vLLM-MindSpore often lacks a clear loading path, causing integration friction for 310P devices.

Core Features & Use Cases

• Supports W8A8SC sparse-quantized models and rank-based weight layout loading.
• Enforces 310P-only execution for sparse paths, with explicit error handling on non-310P devices.
• Provides end-to-end guidance for weight loading, deq_scale handling, and insertion into Quant Linear Sparse layers.
• Useful for developers integrating MindSpore sparse quantization into vLLM inference workflows.

Quick Start

Place the sparse-quantized weights under the rank_* directory and initialize the loader to start inference.