llama-cpp

Build exact llama.cpp launch commands for compatible GGUF models.

Updated May 24, 2026

Search Tags:#quantization #model discovery #gguf #server deployment #llama.cpp #local inference #hugging face

Ready to run

npx skills add https://github.com/anxiety135790/hermes-skills --skill llama-cpp-anxiety135790

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/anxiety135790/hermes-skills/tree/main/mlops/inference/llama-cpp
Command: npx skills add https://github.com/anxiety135790/hermes-skills --skill llama-cpp-anxiety135790

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes references (resource) components.

What problem does it solve?

This Skill helps you run, choose, and troubleshoot local llama.cpp GGUF inference without guessing the right model file, quantization, or launch command.

Core Features & Use Cases

Local Inference Setup: Launch llama.cpp models on CPU, Apple Silicon, CUDA, ROCm, Intel GPUs, or Windows with the right command.
Hugging Face Model Discovery: Find compatible GGUF repos, inspect exact file sizes, and identify the best quantization for your hardware.
Practical Deployment Support: Build server commands, use the OpenAI-compatible API, and apply optimization or troubleshooting guidance for production or edge environments.
Use Case: A developer wants to host a Hugging Face GGUF model locally and needs the exact file, recommended quant, and a working llama-server command.

Quick Start

Ask for the best GGUF model and llama-server command for your Hugging Face repo and hardware profile.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run a GGUF model locally using llama.cpp?▼

Run a GGUF model locally by finding a compatible Hugging Face repository, selecting hardware-aware quantization, and generating the exact llama-server launch command for your CPU or GPU environment.

What is the best way to choose a GGUF quantization for my hardware?▼

Choose a GGUF quantization by matching exact model file sizes and precision levels to your specific hardware profile, ensuring reliable local execution across CPU, Apple Silicon, CUDA, ROCm, or Intel GPU environments.

How do I deploy a llama.cpp server with an OpenAI-compatible API?▼

Deploy a llama.cpp server by generating exact launch commands tailored to your local setup, enabling you to use the OpenAI-compatible API and apply optimization guidance for production or edge environments.

Can I use Python bindings for local inference with llama.cpp?▼

Yes, you can use Python bindings for local inference by configuring the llama-cpp-python library to execute your selected GGUF model files within your specific local hardware environment.

Why does my local GGUF model fail to launch correctly?▼

Local GGUF model launch failures usually occur when the exact filename is missing or the selected quantization mismatches your hardware, requiring hardware-aware quant matching and troubleshooting for reliable execution.

Does llama.cpp work with Hugging Face model repositories for local inference?▼

llama.cpp works with Hugging Face model repositories by discovering compatible GGUF repos, inspecting exact file sizes, and identifying the best quantization to download and run for your local inference setup.