skills.rest
amd
amdOFFICIAL
Explore all skills by this author →
WebsiteGitHub

quark-torch-file2file-quantization

Quantize large safetensors LLM checkpoints file-to-file with low memory usage.

154|27|Updated Jun 19, 2024
Search Tags:#llm#checkpoint#quantization#safetensors#moe#low-memory
One-click install
npx skills add https://github.com/amd/Quark --skill quark-torch-file2file-quantization
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: quark-torch-file2file-quantization
Source: https://github.com/amd/Quark/tree/main/.claude/skills-impl/l1-atomic/torch/quark-torch-file2file-quantization
Command: npx skills add https://github.com/amd/Quark --skill quark-torch-file2file-quantization

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you quantize very large safetensors language models that cannot fit in memory, so you can adapt checkpoints without loading the full model at once.

Core Features & Use Cases

  • Low-Memory File-to-File Quantization: Run direct checkpoint quantization on sharded safetensors with minimal memory usage.
  • Template and Naming Adaptation: Register external LLM templates, handle naming mismatches, and generate one-time conversion or wrapper scripts.
  • Validation-Guided Execution: Verify minimum-scale experiments, shard consistency, scale pairing, and final output correctness before and after full quantization.
  • Use Case: You have a huge MoE checkpoint such as DeepSeek or Qwen in safetensors format and need a safe, low-memory path to produce a quantized output directory.

Quick Start

Ask the assistant to generate or run a low-memory file-to-file quantization plan for your safetensors checkpoint, including template registration, a minimum validation experiment, and the final output manifest.

Frequently Asked Questions about quark-torch-file2file-quantization

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I quantize a large safetensors LLM checkpoint without loading it into memory?▼

Low-memory file-to-file quantization processes sharded safetensors individually, applying weight-name normalization and template adaptation to produce a quantized output directory without loading the full checkpoint.

Can I use file-to-file quantization for MoE models like DeepSeek or Qwen?▼

Yes, this low-memory quantization workflow explicitly supports MoE checkpoints like DeepSeek or Qwen, applying template registration, weight-name normalization, and shard consistency checks for safe file-to-file adaptation.

What's the best way to handle template mismatches when quantizing safetensors checkpoints?▼

Register external LLM templates and generate one-time conversion or wrapper scripts to normalize weight names and resolve template mismatches before running the full checkpoint quantization process.

Do I need to run a validation experiment before quantizing a huge LLM checkpoint?▼

Yes, you must run a minimum-scale validation experiment to verify scale pairing, shard consistency, and output correctness before executing the full file-to-file quantization on your large safetensors model.

Why does safetensors checkpoint quantization fail with out of memory errors?▼

Checkpoint quantization fails with out of memory errors when attempting to load the entire model simultaneously; applying a low-memory file-to-file workflow processes individual shards to avoid this memory constraint.

Related Skills

Explore More Software Engineering Skills→
gar-aigar-ai
1

rust-ml-huggingface-porting

Detect and resolve tensor naming mismatches when porting HuggingFace models to Candle.

Official
Advanced
eaglhuangeaglhuang
3

doc-shard-manager

Partition large Markdown and JSON documents into configurable shard groups.

Community
Advanced
timfogarty1549timfogarty1549

musmem-extract-distinct-athletes

Extract athlete names from MusMem semicolon-delimited files into candidate-group TSVs.

Community
Intermediate
ldzhhxxldzhhxx
1

serving-llms-vllm

Serve OpenAI-compatible LLM APIs with optimized throughput and quantization.

Community
Advanced