skills.rest
faramirezs
faramirezsCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Automate local GGUF inference and model discovery for llama.cpp.

Updated Jun 15, 2026
Search Tags:#server#inference#llama-cpp#model-discovery#gguf#hugging-face
One-click install
npx skills add https://github.com/faramirezs/talking_printer --skill llama-cpp-faramirezs
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/faramirezs/talking_printer/tree/main/hermes-agent/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/faramirezs/talking_printer --skill llama-cpp-faramirezs

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

llama.cpp users need a cohesive workflow to run local GGUF inference and discover Hugging Face models without juggling multiple tools.

Core Features & Use Cases

  • Local GGUF inference on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.
  • Quant selection and repository discovery on Hugging Face Hub for llama.cpp compatible models.
  • Directly launch a llama-server or llama-cli from the Hub snippet or exact GGUF filename and run experiments locally.

Quick Start

Identify a suitable GGUF from HF Hub for your hardware and start a local server or CLI with the chosen repo and quant.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp across different operating systems?▼

Local GGUF inference can be run on Linux, macOS, and Windows by launching a llama-server or llama-cli instance. It supports execution across CPU, Apple Silicon, CUDA, ROCm, and Intel GPUs based on your hardware constraints.

What is the best way to find compatible GGUF models on Hugging Face for my hardware?▼

Discovering compatible GGUF models is done by using Hugging Face hub workflows and the HF tree API to enumerate files, sizes, and quant labels. It checks repository frontmatter to identify llama.cpp compatible models matching your hardware limits.

Can I launch a local llama-server directly from a Hugging Face Hub snippet?▼

Yes, you can launch a local llama-server or llama-cli directly from a Hugging Face Hub snippet or exact GGUF filename. This enables you to run local model experiments immediately after repository discovery.

How does quant selection work when discovering Hugging Face repos for local inference?▼

Quant selection works by enumerating Hugging Face repository files and sizes through the HF tree API. It identifies available quantization labels to ensure the selected GGUF matches your specific hardware constraints for local inference.

Do I need separate tools for model discovery and running llama.cpp inference locally?▼

No, you do not need separate tools for model discovery and local llama.cpp inference. This Skill provides a cohesive workflow that handles both Hugging Face repository discovery and direct local server or CLI execution.

Related Skills

Explore More Software Engineering Skills→
lxt070409lxt070409
1

llama-cpp

Run local inference with llama.cpp models and discover GGUF models from the Hugging Face Hub.

Community
Advanced
x-TheFoxx-TheFox

llama-cpp

Run local GGUF inference across CPU, Apple Silicon, and GPU backends.

Community
Intermediate
SriRamkunamsettySriRamkunamsetty

llama-cpp

Run local GGUF inference and discover Hugging Face models for llama.cpp.

Community
Intermediate
datathingsdatathings
10

llamacpp

Reference the llama.cpp C API for model loading and inference workflows.

Official
Advanced