skills.rest
Andrew-Girgis
Andrew-GirgisCOMMUNITY
Explore all skills by this author →
Website𝕏@AndrewGirgisGitHub

huggingface-local-models

Select, quantize, and run Hugging Face models with llama.cpp.

Updated Jun 15, 2026
Search Tags:#model deployment#huggingface#gguf#llama.cpp#local model
One-click install
npx skills add https://github.com/Andrew-Girgis/token-holdem --skill huggingface-local-models-andrew-girgis
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: huggingface-local-models
Source: https://github.com/Andrew-Girgis/token-holdem/tree/main/.agents/skills/huggingface-local-models
Command: npx skills add https://github.com/Andrew-Girgis/token-holdem --skill huggingface-local-models-andrew-girgis

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill unit provides the instructions to select, prepare, and run local models from the Hugging Face Hub using llama.cpp and GGUF on various hardware configurations.

Core Features & Use Cases

  • Model Selection: Offers a workflow to find compatible llama.cpp models on the Hugging Face Hub.
  • Quantization Options: Assists in selecting the appropriate quantization for model efficiency and compatibility.
  • Running Models: Includes detailed commands for executing models using llama.cpp or llama-server.

Quick Start

Install llama.cpp and authenticate for gated repos. Use hf download <repo> to get models and launch them with llama-cli -hf <repo>:<QUANT> or llama-server -hf <repo>:<QUANT>.

Frequently Asked Questions about huggingface-local-models

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run Hugging Face models locally using llama.cpp?▼

To run Hugging Face models locally, use the `hf download <repo>` command to retrieve the model, then launch it with `llama-cli -hf <repo>:<QUANT>` or `llama-server -hf <repo>:<QUANT>` using llama.cpp.

What quantization options are available for local models?▼

Quantization options for local models determine efficiency and hardware compatibility. You can specify a quantization level using the `:<QUANT>` suffix when executing models via llama-cli or llama-server with GGUF files.

Do I need to authenticate to download gated Hugging Face repos for local deployment?▼

Yes, you must authenticate to access gated repositories on Hugging Face for local deployment. After installing llama.cpp, authenticate your account before using `hf download <repo>` to retrieve restricted models.

Can I use llama.cpp local models across different hardware configurations?▼

llama.cpp local models are designed to work across various hardware configurations. The Skill helps manage quantization and deployment options to ensure compatibility and efficiency on your specific machine.

What is the best way to find compatible llama.cpp models on Hugging Face?▼

The best way to find compatible llama.cpp models is to use the model selection workflow provided, which searches the Hugging Face Hub specifically for GGUF formatted models that run efficiently with llama.cpp.

Related Skills

Explore More Software Engineering Skills→
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
GitnappGitnapp

llama-cpp

Discover GGUF models on Hugging Face and launch llama.cpp inference commands.

Community
Advanced
CyapstayeCyapstaye
9

llama-cpp

Discover GGUF models on Hugging Face and run local inference with llama.cpp.

Community
Advanced
helix4uhelix4u

llama-cpp

Run LLM inference on CPU, Apple Silicon, and non-NVIDIA GPUs with GGUF models.

Community
Intermediate