skills.rest
Mesh-LLM
Mesh-LLMOFFICIAL
Explore all skills by this author →
WebsiteGitHub

hf-quant-and-layer-package-jobs

Runs GGUF quantization and Skippy layer-package creation as one local or Hugging Face Jobs workflow.

3.4k|405|Updated Feb 11, 2026
Search Tags:#gguf#hugging face jobs#model publishing#mesh-llm#skippy#Quantization#layer package
One-click install
npx skills add https://github.com/Mesh-LLM/mesh-llm --skill hf-quant-and-layer-package-jobs
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hf-quant-and-layer-package-jobs
Source: https://github.com/Mesh-LLM/mesh-llm/tree/main/.agents/skills/hf-quant-and-layer-package-jobs
Command: npx skills add https://github.com/Mesh-LLM/mesh-llm --skill hf-quant-and-layer-package-jobs

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Producing a quantized GGUF model and a Skippy layer package from a BF16/FP16 source repo involves many ordered steps, and running them separately risks inconsistent artifacts, failed resumes, and mismatched published repos. This Skill coordinates both phases as one workflow with verification gates between them.

Core Features & Use Cases

  • Combined quantize-and-package workflow: Runs skippy-quantize (init, run, verify) followed by mesh-llm models package, publishing both artifacts to Hugging Face.
  • Local and HF Jobs execution: Supports running the full pipeline locally with mounted repos or as a single Hugging Face Job with read-only source and read/write target volumes.
  • Resume and validation rules: Resumes quantization at the first missing shard, skips quantization when the quant repo verifies, and records commits, selectors, tensor recipes, and job costs for promotion decisions.
  • Use Case: You have a BF16 GGUF repo and need a Q4_K_M quant plus a Skippy layer package for mesh-llm distribution. This Skill dry-runs the quant job, executes it within a memory budget, verifies the output, then packages and publishes both repos.

Quick Start

Ask the agent to quantize your BF16 GGUF repo with skippy-quantize and then create and publish the Skippy layer package as one workflow.

Frequently Asked Questions about hf-quant-and-layer-package-jobs

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I quantize a BF16 GGUF model and create a Skippy layer package in one workflow?▼

Run skippy-quantize init-quant, run-quant, and verify-job against the BF16 source repo, then run mesh-llm models package on the verified quant repo. The workflow can execute locally or as a single Hugging Face Job with mounted volumes.

Which quantization tool should I use for GGUF quantization in this workflow?▼

Use skippy-quantize for the quantization phase. Do not use llama-quantize, llama-quantise, convert_hf_to_gguf.py, or hf_to_gguf.py, as the workflow is built around skippy-quantize manifests and verification.

Can I run quantization and packaging as a Hugging Face Job?▼

Yes. Submit one HF Job that mounts the BF16 source repo read-only and the target quant repo read/write, runs the quant phases, verifies the result, and then runs the package phase, recording both repo commits.

What happens if a quantization run is interrupted?▼

skippy-quantize resumes at the first missing shard if quant shards already exist. If the quant repo verifies successfully, quantization is skipped entirely and only the package job runs or is inspected.

Should I delete a verified quant repo to force a clean package run?▼

No. A verified quant repo is the source of truth for packaging and must not be deleted. Package jobs consume the published quant artifact directly.

Related Skills

Explore More Software Engineering Skills→
RachasumanthRachasumanth

model-publisher

Package trained AI models and publish them to Hugging Face Hub.

Community
Advanced
LynxLabVNLynxLabVN

serving-llms-vllm

Serve large language models with OpenAI-compatible APIs and quantization.

Official
Advanced
NVIDIANVIDIA
3.7k

qad

Runs ModelOpt Quantization-Aware Distillation on Slurm through Megatron Bridge to recover PTQ accuracy gaps.

Official
Advanced
davpatel605-beepdavpatel605-beep

llama-cpp

Configure llama.cpp deployments and select GGUF quantizations for local inference.

Community
Advanced