llama-cpp

Enable local GGUF inference and model discovery workflows for llama.cpp.

1|Updated Apr 21, 2026

Search Tags:#server #inference #cpu #model-discovery #gguf #llama.cpp #hugging-face

Ready to run

npx skills add https://github.com/ChangZhou-xj/zxj_skill --skill llama-cpp-changzhou-xj

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/ChangZhou-xj/zxj_skill/tree/main/mlops/inference/llama-cpp
Command: npx skills add https://github.com/ChangZhou-xj/zxj_skill --skill llama-cpp-changzhou-xj

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

llama.cpp local GGUF inference + HF Hub model discovery.

Core Features & Use Cases

Local GGUF inference across CPU, Apple Silicon, CUDA, ROCm, and various GPUs.
Model discovery on Hugging Face Hub to identify GGUF files and compatible repos.
Guided workflows to build llama-server or llama-cli commands from Hub data and to enumerate available GGUFs for quant selection.

Quick Start

Install llama.cpp and run a local GGUF workflow by discovering HF repos and launching a server or CLI.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run local GGUF inference using llama.cpp?▼

Local GGUF inference in llama.cpp is enabled by discovering Hugging Face Hub repos and building llama-server or llama-cli commands to execute models directly on your hardware backend.

Can I run GGUF models on Apple Silicon or CUDA GPUs?▼

Yes, GGUF inference is supported across CPU, Apple Silicon, CUDA, ROCm, and various other GPU backends, allowing flexible hardware acceleration for local model execution.

How do I find compatible GGUF models on the Hugging Face Hub?▼

Model discovery on the Hugging Face Hub identifies compatible repos and enumerates available GGUF files, guiding you through quant selection to find the right model for your setup.

What is the best way to select a quantization level for a GGUF file?▼

Selecting a GGUF quant involves enumerating available model files from the Hugging Face Hub, guiding you to balance inference quality and memory usage for your specific CPU or GPU.

Do I need an internet connection to use llama.cpp for local inference?▼

An internet connection is needed for Hugging Face Hub model discovery to identify and download GGUF files, but the subsequent local GGUF inference runs entirely offline on your machine.

How do I build llama-server commands from Hugging Face model data?▼

Guided workflows take Hugging Face Hub repo data and help you construct llama-server or llama-cli commands, ensuring the correct parameters are set for your chosen backend and quantization.