skills.rest
anxiety135790
anxiety135790COMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Build exact llama.cpp launch commands for compatible GGUF models.

Updated May 24, 2026
Search Tags:#quantization#model discovery#gguf#server deployment#llama.cpp#local inference#hugging face
One-click install
npx skills add https://github.com/anxiety135790/hermes-skills --skill llama-cpp-anxiety135790
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/anxiety135790/hermes-skills/tree/main/mlops/inference/llama-cpp
Command: npx skills add https://github.com/anxiety135790/hermes-skills --skill llama-cpp-anxiety135790

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes references (resource) components.

What problem does it solve?

This Skill helps you run, choose, and troubleshoot local llama.cpp GGUF inference without guessing the right model file, quantization, or launch command.

Core Features & Use Cases

  • Local Inference Setup: Launch llama.cpp models on CPU, Apple Silicon, CUDA, ROCm, Intel GPUs, or Windows with the right command.
  • Hugging Face Model Discovery: Find compatible GGUF repos, inspect exact file sizes, and identify the best quantization for your hardware.
  • Practical Deployment Support: Build server commands, use the OpenAI-compatible API, and apply optimization or troubleshooting guidance for production or edge environments.
  • Use Case: A developer wants to host a Hugging Face GGUF model locally and needs the exact file, recommended quant, and a working llama-server command.

Quick Start

Ask for the best GGUF model and llama-server command for your Hugging Face repo and hardware profile.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run a GGUF model locally using llama.cpp?▼

Run a GGUF model locally by finding a compatible Hugging Face repository, selecting hardware-aware quantization, and generating the exact llama-server launch command for your CPU or GPU environment.

What is the best way to choose a GGUF quantization for my hardware?▼

Choose a GGUF quantization by matching exact model file sizes and precision levels to your specific hardware profile, ensuring reliable local execution across CPU, Apple Silicon, CUDA, ROCm, or Intel GPU environments.

How do I deploy a llama.cpp server with an OpenAI-compatible API?▼

Deploy a llama.cpp server by generating exact launch commands tailored to your local setup, enabling you to use the OpenAI-compatible API and apply optimization guidance for production or edge environments.

Can I use Python bindings for local inference with llama.cpp?▼

Yes, you can use Python bindings for local inference by configuring the llama-cpp-python library to execute your selected GGUF model files within your specific local hardware environment.

Why does my local GGUF model fail to launch correctly?▼

Local GGUF model launch failures usually occur when the exact filename is missing or the selected quantization mismatches your hardware, requiring hardware-aware quant matching and troubleshooting for reliable execution.

Does llama.cpp work with Hugging Face model repositories for local inference?▼

llama.cpp works with Hugging Face model repositories by discovering compatible GGUF repos, inspecting exact file sizes, and identifying the best quantization to download and run for your local inference setup.

Related Skills

Explore More Software Engineering Skills→
webdevtodayjasonwebdevtodayjason

llama-cpp

Automate local GGUF inference and model discovery for llama.cpp.

Community
Advanced
GitnappGitnapp

llama-cpp

Discover GGUF models on Hugging Face and launch llama.cpp inference commands.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
SignmanalSignmanal
1

llama-cpp

Run local GGUF model inference with llama.cpp across CPU and GPU hardware.

Community
Intermediate