skills.rest
huggingface
huggingfaceOFFICIAL
Explore all skills by this author →
Website𝕏@huggingfaceGitHub

huggingface-local-models

Locate and run Hugging Face GGUF models locally with llama.cpp.

78|18|Updated Apr 17, 2026
Search Tags:#conversion#huggingface#local-models#model-discovery#gguf#llama.cpp#local-serving
One-click install
npx skills add https://github.com/huggingface/context-course --skill huggingface-local-models-huggingface
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: huggingface-local-models
Source: https://github.com/huggingface/context-course/tree/main/projects/inference/.agents/skills/huggingface-local-models
Command: npx skills add https://github.com/huggingface/context-course --skill huggingface-local-models-huggingface

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Locate and run Hugging Face GGUF models locally using llama.cpp across CPU, Metal, CUDA, or ROCm.

Core Features & Use Cases

  • Model discovery: Find llama.cpp-compatible GGUF models on Hugging Face.
  • Quant and deployment: Recommend exact quant, verify files, and launch local servers.
  • Local serving: OpenAI-compatible local serving workflows and quick testing.

Quick Start

Install and configure llama.cpp, then identify a suitable GGUF model and start a local server.

Frequently Asked Questions about huggingface-local-models

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run Hugging Face GGUF models locally using llama.cpp?▼

Run Hugging Face GGUF models locally by discovering llama.cpp-compatible files, selecting an exact quant, verifying files, and launching a local server across CPU, Metal, CUDA, or ROCm.

How do I find the exact GGUF file and quant for local model deployment?▼

Find the exact GGUF file and quant by using model discovery to locate llama.cpp-compatible models on Hugging Face and applying quant guidance to recommend the optimal deployment file.

Does llama.cpp support local serving on CUDA, Metal, and ROCm?▼

Yes, llama.cpp supports local serving on CUDA, Metal, and ROCm, as well as CPU. This skill configures the local server launch across these hardware backends for robust local deployment.

Can I convert Hugging Face models to GGUF format for local serving?▼

Yes, you can convert models to GGUF format for local serving. The skill provides optional conversion capabilities alongside model discovery and server launch workflows for deployment.

What is the best way to set up an OpenAI-compatible local server with GGUF models?▼

The best way to set up an OpenAI-compatible local server is to locate a suitable GGUF model on Hugging Face, verify its files, and launch the local server using llama.cpp workflows.

Related Skills

Explore More Software Engineering Skills→
tangzheng202202tangzheng202202
1

llama-cpp

Execute CPU-based LLM inference with GGUF quantization on Apple Silicon and non-NVIDIA GPUs.

Community
Intermediate
InverterNetworkInverterNetwork

llama-cpp

Discovers GGUF models from Hugging Face Hub and configures llama.cpp inference servers.

Official
Advanced
SriRamkunamsettySriRamkunamsetty

llama-cpp

Run local GGUF inference and discover Hugging Face models for llama.cpp.

Community
Intermediate
SignmanalSignmanal
1

llama-cpp

Run local GGUF model inference with llama.cpp across CPU and GPU hardware.

Community
Intermediate