skills.rest
ChangZhou-xj
ChangZhou-xjCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Enable local GGUF inference and model discovery workflows for llama.cpp.

1|Updated Apr 21, 2026
Search Tags:#server#inference#cpu#model-discovery#gguf#llama.cpp#hugging-face
One-click install
npx skills add https://github.com/ChangZhou-xj/zxj_skill --skill llama-cpp-changzhou-xj
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/ChangZhou-xj/zxj_skill/tree/main/mlops/inference/llama-cpp
Command: npx skills add https://github.com/ChangZhou-xj/zxj_skill --skill llama-cpp-changzhou-xj

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

llama.cpp local GGUF inference + HF Hub model discovery.

Core Features & Use Cases

  • Local GGUF inference across CPU, Apple Silicon, CUDA, ROCm, and various GPUs.
  • Model discovery on Hugging Face Hub to identify GGUF files and compatible repos.
  • Guided workflows to build llama-server or llama-cli commands from Hub data and to enumerate available GGUFs for quant selection.

Quick Start

Install llama.cpp and run a local GGUF workflow by discovering HF repos and launching a server or CLI.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp?▼

Local GGUF inference in llama.cpp is enabled by discovering Hugging Face Hub repos and building llama-server or llama-cli commands to execute models directly on your hardware backend.

Can I run GGUF models on Apple Silicon or CUDA GPUs?▼

Yes, GGUF inference is supported across CPU, Apple Silicon, CUDA, ROCm, and various other GPU backends, allowing flexible hardware acceleration for local model execution.

How do I find compatible GGUF models on the Hugging Face Hub?▼

Model discovery on the Hugging Face Hub identifies compatible repos and enumerates available GGUF files, guiding you through quant selection to find the right model for your setup.

What is the best way to select a quantization level for a GGUF file?▼

Selecting a GGUF quant involves enumerating available model files from the Hugging Face Hub, guiding you to balance inference quality and memory usage for your specific CPU or GPU.

Do I need an internet connection to use llama.cpp for local inference?▼

An internet connection is needed for Hugging Face Hub model discovery to identify and download GGUF files, but the subsequent local GGUF inference runs entirely offline on your machine.

How do I build llama-server commands from Hugging Face model data?▼

Guided workflows take Hugging Face Hub repo data and help you construct llama-server or llama-cli commands, ensuring the correct parameters are set for your chosen backend and quantization.

Related Skills

Explore More Software Engineering Skills→
Tyler-R-KendrickTyler-R-Kendrick
1

huggingface-local-models

Search Hugging Face Hub and launch GGUF models via llama.cpp.

Community
Intermediate
zengbaochengzengbaocheng

llama-cpp

Run local GGUF inference with llama.cpp using llama-server or llama-cli.

Community
Advanced
vikrant-projectvikrant-project
2

llama-cpp

Execute GGUF inference locally with llama.cpp and Python bindings.

Community
Intermediate
quazfentonquazfenton

cli-anything-ollama

Manage local Ollama models, generate text, and create embeddings via CLI.

Community
Intermediate