skills.rest
BermudaLocals
BermudaLocalsCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

llama-cpp

Execute local GGUF inference and discover Hugging Face models with llama.cpp.

1|1|Updated Apr 26, 2026
Search Tags:#inference#model-discovery#gguf#llama.cpp#hugging-face#server-cli
One-click install
npx skills add https://github.com/BermudaLocals/hermes-agent-lite --skill llama-cpp-bermudalocals
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/BermudaLocals/hermes-agent-lite/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/BermudaLocals/hermes-agent-lite --skill llama-cpp-bermudalocals

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0.

What problem does it solve?

This Skill enables local GGUF inference with llama.cpp and streamlines model discovery on Hugging Face, letting you run models offline and locate compatible GGUFs quickly.

Core Features & Use Cases

  • Run local GGUF inference on CPU or GPU for llama.cpp-based models.
  • Discover Hugging Face GGUF repositories and exact files for quick deployment with llama-server or llama-cli.
  • Build and test server/cli workflows from HF hub pages and tree API.

Quick Start

Launch a local llama.cpp workflow to discover GGUF models and run inference on a chosen Hugging Face repo.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp on my hardware?▼

Run local GGUF inference by executing CPU or GPU-accelerated generation directly through llama.cpp, which supports offline model execution on your supported hardware without external API dependencies.

Can I discover compatible GGUF models on Hugging Face for llama-server deployment?▼

Discover compatible GGUF models on Hugging Face by locating exact repository files and generating ready-to-use llama-server or llama-cli commands from HF hub pages or tree API structures.

What is quantization-aware model discovery and when do I need it for local inference?▼

Quantization-aware model discovery validates GGUF files against supported hardware constraints, ensuring you select compatible quantized models for local llama.cpp inference before downloading large repository files.

Does llama-cpp-python support building workflows from Hugging Face hub URLs?▼

Yes, llama-cpp-python supports building and testing server or CLI workflows directly from Hugging Face hub URLs and tree API validation, streamlining the transition from model discovery to execution.

What are the limitations of running offline GGUF inference with llama.cpp?▼

Offline GGUF inference with llama.cpp is limited by your local hardware capabilities for CPU or GPU acceleration, requiring sufficient memory to load the selected quantized model entirely on your machine.

Related Skills

Explore More Software Engineering Skills→
PiaoxuemoliPiaoxuemoli

llama-cpp

Run local GGUF inference with llama.cpp models from Hugging Face Hub.

Community
Advanced
ovachieverovachiever
52

llama-factory

Automate LLaMA-family model fine-tuning via the Llama-Factory WebUI.

Community
Advanced
SriRamkunamsettySriRamkunamsetty

llama-cpp

Run local GGUF inference and discover Hugging Face models for llama.cpp.

Community
Intermediate
juliuss1907juliuss1907

llama-cpp

Run local GGUF inference with llama.cpp models from Hugging Face Hub.

Community
Advanced