skills.rest
AlexiosBluffMara
AlexiosBluffMaraCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local GGUF inference and discover models via Hugging Face Hub.

2|Updated Apr 25, 2026
Search Tags:#server#inference#huggingface#llama-cpp#model-discovery#gguf
One-click install
npx skills add https://github.com/AlexiosBluffMara/mercury --skill llama-cpp-alexiosbluffmara
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/AlexiosBluffMara/mercury/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/AlexiosBluffMara/mercury --skill llama-cpp-alexiosbluffmara

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0.

What problem does it solve?

llama.cpp enables fast, offline inference and local model discovery by enabling local GGUF inference and Hub-based model discovery workflow.

Core Features & Use Cases

  • Local GGUF inference across CPU and GPUs
  • Hugging Face Hub model discovery and launching llama-server or llama-cli workflows
  • Hardware-aware quantization and deployment for edge devices and servers

Quick Start

Start by selecting a local GGUF model and spin up a llama-server with the recommended quant for your hardware.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama-cpp-python?▼

Run local GGUF inference by selecting a local model and spinning up a llama-server or llama-cli workflow with the recommended quantization for your hardware.

Can I use llama.cpp for inference on Apple Silicon and CUDA GPUs?▼

Yes, llama.cpp supports CPU and GPU deployments across Apple Silicon, CUDA, ROCm, and AMD/Intel GPUs for hardware-aware quantization and inference.

What is the best way to discover GGUF models on Hugging Face Hub?▼

Discover GGUF models by using the Hugging Face Hub plus local-app/tree API to find matching GGUF files suitable for your local inference deployment.

Do I need llama-cpp-python to run a llama-server workflow?▼

Yes, you need llama-cpp-python version 0.2.0 or higher for Python bindings and integration to run llama-server or llama-cli workflows.

Does llama.cpp support hardware-aware quantization for edge devices?▼

Yes, llama.cpp provides hardware-aware quantization and deployment tailored for both edge devices and servers across various CPU and GPU architectures.

Why use GGUF format for local offline inference?▼

GGUF format enables fast offline inference and local model discovery, allowing you to run models directly on your hardware without requiring external API connections.

Related Skills

Explore More Software Engineering Skills→
peteromalletpeteromallet
97

llama-cpp

Run CPU-only LLM inference with llama-cpp on non-NVIDIA hardware.

Community
Intermediate
AissenLiuAissenLiu

llama-cpp

Run GGUF-based llama.cpp inference locally and discover compatible Hugging Face models.

Community
Intermediate
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
jackspacejackspace
16

arboreto

Infer gene regulatory networks from expression data using GRNBoost2 or GENIE3.

Community
Advanced