skills.rest
x-TheFox
x-TheFoxCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local GGUF inference across CPU, Apple Silicon, and GPU backends.

Updated Apr 27, 2026
Search Tags:#quantization#inference#cpu#gguf#apple-silicon#llama.cpp#hugging-face
One-click install
npx skills add https://github.com/x-TheFox/Corvus --skill llama-cpp-x-thefox
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/x-TheFox/Corvus/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/x-TheFox/Corvus --skill llama-cpp-x-thefox

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0.

What problem does it solve?

Provides local GGUF inference and model discovery workflows for llama.cpp, enabling efficient on-device deployment and HF Hub model discovery.

Core Features & Use Cases

  • Local GGUF inference across CPU, Apple Silicon, and GPU backends
  • Quantization selection and Hugging Face repo discovery for llama.cpp
  • HF Hub integration to locate and start servers from pre-quantized GGUFs

Quick Start

Install llama.cpp and run the llama-server or llama-cli with a GGUF model from Hugging Face to start local inference.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp on Apple Silicon?▼

Local GGUF inference on Apple Silicon is supported alongside CPU and GPU backends. You can start the llama-server or llama-cli with a pre-quantized GGUF model from Hugging Face.

What is GGUF quantization and when do I need it for Hugging Face models?▼

GGUF quantization compresses model weights for efficient on-device deployment. You need it when downloading pre-quantized models from the Hugging Face Hub to run local inference without severe performance degradation.

Can I use llama-cpp-python to discover and download models from the HF Hub?▼

Yes, you can use Hugging Face repo discovery to locate and start servers from pre-quantized GGUFs. This requires the llama-cpp-python dependency to interface with the llama.cpp ecosystem.

What is the best way to select a quantization level for local llama.cpp deployment?▼

Selecting the right quantization involves balancing memory constraints and inference quality. This workflow provides quantization selection guidance to optimize local GGUF deployment across your target hardware.

Do I need to build llama.cpp from source to run local inference?▼

Building from the Hugging Face Hub is supported, but you can also run pre-built llama-server or llama-cli commands. You must install required dependencies like llama-cpp-python to execute local inference workflows.

Why does local GGUF inference fail without the correct llama.cpp backend setup?▼

Local GGUF inference requires proper backend configuration for CPU, Apple Silicon, or GPU. Without the correct llama.cpp ecosystem setup and dependencies, the server cannot locate or execute the model files.

Related Skills

Explore More Software Engineering Skills→
webdevtodayjasonwebdevtodayjason

llama-cpp

Automate local GGUF inference and model discovery for llama.cpp.

Community
Advanced
diskrotrepodiskrotrepo
9

yulan-mini

Generate song lyrics, audio prompts, and chat completions via a llama.cpp server.

Official
Intermediate
vikrant-projectvikrant-project
2

llama-cpp

Execute GGUF inference locally with llama.cpp and Python bindings.

Community
Intermediate
Ced3-hanCed3-han

gguf-quantization

Convert HuggingFace models to GGUF format and quantize them for llama.cpp inference.

Community
Advanced