skills.rest
jleechanorg
jleechanorgOFFICIAL
Explore all skills by this author →
GitHub

llama-cpp

Automate local GGUF inference and Hugging Face model discovery for llama.cpp.

Updated Jun 28, 2026
Search Tags:#server#quantization#inference#huggingface#model-discovery#gguf#llama.cpp
One-click install
npx skills add https://github.com/jleechanorg/hermes-agent --skill llama-cpp-jleechanorg
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/jleechanorg/hermes-agent/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/jleechanorg/hermes-agent --skill llama-cpp-jleechanorg

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

llama.cpp is a popular local inference framework, but users struggle to discover compatible GGUF models and run local servers efficiently. This Skill provides a structured workflow to locate GGUF files on Hugging Face, select appropriate quantizations, and launch local inference via llama-server or llama-cli.

Core Features & Use Cases

  • Local GGUF inference: run CPU or GPU-accelerated models on desktop or server environments.
  • Model discovery: search Hugging Face Hub for llama.cpp-compatible GGUF models and map them to exact filenames.
  • Quantization guidance: identify HF-recommended quant labels and construct exact commands for local servers.

Quick Start

Install llama.cpp, discover compatible GGUF models on Hugging Face, and start a local server or CLI for inference.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp?▼

To run local GGUF inference, you can use this Skill to start llama-server or llama-cli. It automates quantization selection and command construction for CPU, Apple Silicon, CUDA, and ROCm environments.

How do I find compatible GGUF models on Hugging Face?▼

Finding compatible GGUF models on Hugging Face is handled through repository exploration using the tree API. The Skill maps HF-recommended quant labels to exact filenames for precise downloads.

Does llama.cpp inference work on Apple Silicon and CUDA GPUs?▼

Yes, llama.cpp inference works across diverse hardware including Apple Silicon and CUDA GPUs. This Skill provides hardware-specific guidance for launching local servers and running accelerated inference.

What's the best way to select the right quantization for a GGUF model?▼

The best way to select the right quantization is mapping HF-recommended quant labels to exact GGUF filenames. This Skill uses a local-app snippet and tree API to identify and construct exact commands for the chosen quant.

How do I start a local llama-server for Hugging Face models?▼

You can start a local llama-server by locating the target GGUF file on Hugging Face and generating the exact startup command. This Skill automates the server and CLI startup process for discovered repositories.

Related Skills

Explore More Software Engineering Skills→
fikriaffikriaf

llama-cpp

Automate discovery, download, and local inference of llama.cpp models from Hugging Face Hub.

Community
Advanced
jayll1303jayll1303
17

llama-cpp-inference

Run local GGUF model inference and OpenAI-compatible serving with llama.cpp.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
CyapstayeCyapstaye
9

llama-cpp

Discover GGUF models on Hugging Face and run local inference with llama.cpp.

Community
Advanced