skills.rest
zengbaocheng
zengbaochengCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local GGUF inference with llama.cpp using llama-server or llama-cli.

Updated May 23, 2026
Search Tags:#huggingface#model-discovery#gguf#llama.cpp#local-inference
One-click install
npx skills add https://github.com/zengbaocheng/hermes-tech-hub --skill llama-cpp-zengbaocheng
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/zengbaocheng/hermes-tech-hub/tree/main/mlops/inference/llama-cpp
Command: npx skills add https://github.com/zengbaocheng/hermes-tech-hub --skill llama-cpp-zengbaocheng

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This skill enables local GGUF inference and model discovery for llama.cpp, reducing latency by avoiding remote API calls and enabling offline workflows.

Core Features & Use Cases

  • Local GGUF inference on CPU or GPU with llama.cpp and GGUF models from Hugging Face Hub.
  • Quant selection and model discovery via HF repo pages and tree API to identify compatible GGUF files.
  • Quick-startable deployment with llama-server or llama-cli for rapid experimentation.

Quick Start

Install llama.cpp, pick a GGUF model from HF, and start a local server or CLI for inference.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp to avoid remote API calls?▼

Run local GGUF inference by processing a local model file with llama-server or llama-cli. This avoids remote API calls, reduces latency, and enables offline workflows using Hugging Face Hub models.

How do I find compatible GGUF files on Hugging Face for local inference?▼

Find compatible GGUF files for local inference by querying the Hugging Face Hub repository pages and tree API. This model discovery process helps identify specific quantization labels and exact files to run.

Can I use llama-server and llama-cli for rapid experimentation with Hugging Face models?▼

Yes, you can use llama-server or llama-cli for quick-startable deployment and rapid experimentation. They enable local GGUF inference on CPU or GPU with models sourced directly from the Hugging Face Hub.

What's the best way to select a quantization level for a GGUF model?▼

Select quantization by identifying compatible GGUF files via HF repo pages and the tree API. You need an exact HF repo and quant label or a specific file to satisfy the local inference workflow.

Does local GGUF inference work for offline workflows without internet access?▼

Yes, local GGUF inference works for offline workflows by avoiding remote API calls. Once you download a model from Hugging Face, you can run it locally to reduce latency and operate without an internet connection.

Related Skills

Explore More Software Engineering Skills→
eliottbusinesseliottbusiness

llama-cpp

Run llama.cpp GGUF model inference locally with Hugging Face Hub discovery.

Community
Advanced
x-TheFoxx-TheFox

llama-cpp

Run local GGUF inference across CPU, Apple Silicon, and GPU backends.

Community
Intermediate
faramirezsfaramirezs

llama-cpp

Automate local GGUF inference and model discovery for llama.cpp.

Community
Advanced
BermudaLocalsBermudaLocals
1

llama-cpp

Execute local GGUF inference and discover Hugging Face models with llama.cpp.

Community
Intermediate