skills.rest
domattioli
domattioliCOMMUNITY
Explore all skills by this author →
GitHub

huggingface-local-models

Locate, download, and run Hugging Face GGUF models with llama.cpp.

Updated Aug 27, 2026
Search Tags:#model-selection#huggingface#local-models#gguf#llama.cpp#local-serving
One-click install
npx skills add https://github.com/domattioli/DomI --skill huggingface-local-models
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: huggingface-local-models
Source: https://github.com/domattioli/DomI/tree/main/plugins/huggingface-skills/skills/huggingface-local-models
Command: npx skills add https://github.com/domattioli/DomI --skill huggingface-local-models

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Locate, download, and run Hugging Face GGUF-based models locally using llama.cpp across CPU and accelerators, simplifying hardware-aware model deployment.

Core Features & Use Cases

  • HF local-app guidance: leverage the local-app hardware compatibility data to pick the right quant.
  • Model discovery & lookup: search Hugging Face for llama.cpp-compatible GGUFs and verify exact filenames.
  • Local serving: launch with llama-cli or llama-server using the exact GGUF file for low-latency inference.

Quick Start

Install the required tools and select a local GGUF model from Hugging Face, then start the server with the chosen GGUF file.

Frequently Asked Questions about huggingface-local-models

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run Hugging Face GGUF models locally using llama.cpp?▼

Run Hugging Face GGUF models locally by searching the API for exact filenames, downloading the files, and launching inference with llama-cli or llama-server for low-latency local serving.

How do I choose the right GGUF quantization for my local hardware?▼

Choose the right GGUF quantization by leveraging Hugging Face local-app hardware compatibility data, which guides model selection to match your CPU and accelerator capabilities.

Can I serve GGUF models locally with hardware acceleration across different devices?▼

Serve GGUF models with hardware acceleration across CPU and accelerators by using llama.cpp to enable offline testing, rapid prototyping, and low-latency inference in diverse environments.

What is the best way to find exact llama.cpp-compatible GGUF files on Hugging Face?▼

The best way to find exact llama.cpp-compatible GGUF files is using the Hugging Face API to search for specific models and verify their exact filenames before downloading.

Do I need an internet connection to run local GGUF models after downloading them?▼

You do not need an internet connection to run local GGUF models after downloading them, because llama.cpp enables fully offline testing and hardware-accelerated serving.

How do I start a local inference server with a downloaded GGUF model?▼

Start a local inference server by executing llama-server with the exact path to your downloaded GGUF file, enabling low-latency local serving for rapid prototyping.

Related Skills

Explore More Software Engineering Skills→
datathingsdatathings
10

llamacpp

Reference the llama.cpp C API for model loading and inference workflows.

Official
Advanced
x-TheFoxx-TheFox

llama-cpp

Run local GGUF inference across CPU, Apple Silicon, and GPU backends.

Community
Intermediate
vikrant-projectvikrant-project
2

llama-cpp

Execute GGUF inference locally with llama.cpp and Python bindings.

Community
Intermediate
SignmanalSignmanal
1

llama-cpp

Run local GGUF model inference with llama.cpp across CPU and GPU hardware.

Community
Intermediate