skills.rest
sheawinkler
sheawinklerCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

llama-cpp

Run local GGUF inference and discover Hugging Face Hub models with llama.cpp.

78|16|Updated Apr 23, 2026
Search Tags:#inference#llama-cpp#model-discovery#gguf#server-deployment#hugging-face
One-click install
npx skills add https://github.com/sheawinkler/hermes-agent-ultra --skill llama-cpp-sheawinkler
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/sheawinkler/hermes-agent-ultra/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/sheawinkler/hermes-agent-ultra --skill llama-cpp-sheawinkler

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes references (resource) components.

What problem does it solve?

Enables fast, local GGUF inference using llama.cpp and simplifies discovering compatible Hugging Face Hub models without relying on cloud-only workflows.

Core Features & Use Cases

  • Local GGUF inference across CPU and GPU backends (GGML, CUDA) with optional offload support.
  • Hugging Face Hub model discovery and quantization-aware selection via the local-app workflow.
  • Quick server deployment with llama-server/llama-cli for testing and production-like iterations.

Quick Start

Install llama.cpp and use the local-app workflow to discover a GGUF model and start a local server.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference without relying on cloud services?▼

Local GGUF inference is enabled by using llama.cpp with CPU or GPU backends like CUDA. It allows offline text generation and quick server deployment via llama-server without depending on cloud-only workflows.

Can I use CUDA GPU acceleration with llama.cpp for local inference?▼

Yes, local inference with llama.cpp supports CUDA GPU backends and optional layer offloading. This hardware-aware approach allows faster local GGUF inference compared to CPU-only processing.

What is the best way to discover compatible GGUF models on Hugging Face Hub?▼

The best way to discover compatible GGUF models is using the local-app workflow for quantization-aware selection. It simplifies finding Hugging Face Hub models optimized for your local llama.cpp environment.

Do I need llama-cpp-python to deploy a local inference server?▼

Yes, llama-cpp-python Python bindings are required to enable local GGUF inference and model discovery. You need this dependency alongside a compatible llama.cpp environment to deploy the local server.

How does llama-server handle testing and production-like iterations?▼

llama-server provides quick server deployment for local GGUF inference, enabling testing and production-like iterations. It operates offline, leveraging llama.cpp to serve models directly from your local hardware.

Related Skills

Explore More Software Engineering Skills→
eliottbusinesseliottbusiness

llama-cpp

Run llama.cpp GGUF model inference locally with Hugging Face Hub discovery.

Community
Advanced
GitnappGitnapp

llama-cpp

Discover GGUF models on Hugging Face and launch llama.cpp inference commands.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
davpatel605-beepdavpatel605-beep

llama-cpp

Configure llama.cpp deployments and select GGUF quantizations for local inference.

Community
Advanced