skills.rest
eliottbusiness
eliottbusinessCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run llama.cpp GGUF model inference locally with Hugging Face Hub discovery.

Updated May 3, 2026
Search Tags:#GGUF#Hugging Face Hub#llama.cpp#model inference#local computing
One-click install
npx skills add https://github.com/eliottbusiness/DeptFlow-Agent --skill llama-cpp-eliottbusiness
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/eliottbusiness/DeptFlow-Agent/tree/main/profile/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/eliottbusiness/DeptFlow-Agent --skill llama-cpp-eliottbusiness

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides local GGUF inference capabilities for llama.cpp and facilitates discovery of models from the Hugging Face Hub.

Core Features & Use Cases

  • Local Inference: Run llama.cpp models on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.
  • Model Discovery: Search and select GGUF files from Hugging Face Hub for llama.cpp.
  • Use Case: A user looking for a llama.cpp model with specific quantization and size constraints can use this Skill to quickly find and download the appropriate model.

Quick Start

To start using this skill, first install llama.cpp and then run the command: llama-server -hf <repo>:<QUANT>

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp?▼

Local GGUF inference is enabled by executing the llama-server command with a Hugging Face Hub repository and quantization parameter. This allows you to run models directly on your local hardware using the specified GGUF files.

Can I run llama.cpp models on Apple Silicon or CUDA GPUs?▼

Yes, llama.cpp inference supports execution across CPU, Apple Silicon, CUDA, ROCm, and Intel GPUs. This hardware compatibility ensures you can run local model inference on your preferred computing architecture.

How do I search for and download GGUF files from Hugging Face Hub?▼

You can discover and select GGUF files from Hugging Face Hub by specifying the repository and quantization format. This allows you to quickly find and download models matching your specific size and quantization constraints.

Do I need Python bindings to use llama.cpp for local model inference?▼

Yes, the llama-cpp-python binding version 0.2.0 or higher is required for local model inference. This dependency provides the necessary interface to run llama.cpp applications and access Hugging Face Hub.

What is the best way to run a llama.cpp model with specific quantization constraints?▼

The best way is to use the Hugging Face Hub model discovery feature to filter by size and quantization. You can then execute the downloaded GGUF file locally using the llama-server command with your chosen hardware configuration.

Why does local GGUF inference require Hugging Face Hub access?▼

Hugging Face Hub access is required to search, select, and download the appropriate GGUF files for your llama.cpp applications. It provides the model discovery mechanism needed to find files matching your specific quantization requirements.

Related Skills

Explore More Software Engineering Skills→
ToqsickToqsick

llama-cpp

Run local GGUF inference for llama.cpp models with quantization selection and Hugging Face Hub discovery.

Community
Advanced
yanochka11yanochka11

llama-cpp

Select GGUF models and generate llama.cpp commands for local inference.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
MesferAliMesferAli

gguf-quantization

Convert HuggingFace models to GGUF and quantize them for llama.cpp inference.

Community
Advanced