skills.rest
whichguy
whichguyCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run llama.cpp GGUF model inference locally on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.

1|Updated Jun 23, 2026
Search Tags:#GGUF#Hugging Face Hub#llama.cpp#local deployment#model inference
One-click install
npx skills add https://github.com/whichguy/hermes-skills-marketplace --skill llama-cpp-whichguy
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/whichguy/hermes-skills-marketplace/tree/main/skills/llama-cpp
Command: npx skills add https://github.com/whichguy/hermes-skills-marketplace --skill llama-cpp-whichguy

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python, ggml, torch, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill provides local GGUF inference capabilities and facilitates discovery of Hugging Face Hub models compatible with llama.cpp, enabling users to run local models on various hardware platforms and explore models from the Hub.

Core Features & Use Cases

  • Local GGUF Inference: Run llama.cpp models locally on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.
  • Model Discovery: Search and select models from the Hugging Face Hub that support llama.cpp.
  • Quick Setup: Install llama.cpp with a simple command or directly from the Hub.
  • Configuration Options: Adjust settings like enabled flags and model configurations.

Quick Start

Install llama.cpp:

brew install llama.cpp

Run a local model:

llama-server -hf bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF model inference using llama.cpp?▼

Local GGUF inference runs directly through llama.cpp commands like llama-server with a Hugging Face Hub model identifier. Install llama.cpp and specify the model repository and quantization tag to execute models locally.

Can I run llama.cpp models on Apple Silicon or CUDA GPUs?▼

Yes, llama.cpp inference supports Apple Silicon, CUDA, ROCm, and Intel GPUs. The Skill parses GGUF files and configures model commands to execute across these hardware platforms.

What is the best way to find Hugging Face Hub models compatible with llama.cpp?▼

The best way is using the Skill's Hugging Face Hub model discovery to search for repositories providing GGUF files. It facilitates selecting compatible models for quick local inference setup without manual format conversion.

Do I need Python and llama-cpp-python to parse GGUF files locally?▼

Yes, Python and llama-cpp-python are required to parse GGUF files and execute model commands locally. The Skill relies on these dependencies alongside ggml and torch to facilitate local inference.

Does local GGUF inference require installing llama.cpp separately before running models?▼

Yes, installing llama.cpp separately is required before running models, achievable via a simple command like brew install llama.cpp. This provides the foundational executable needed to parse GGUF files and start inference.

Related Skills

Explore More Software Engineering Skills→
AlexiosBluffMaraAlexiosBluffMara
2

llama-cpp

Run local GGUF inference and discover models via Hugging Face Hub.

Community
Intermediate
ovachieverovachiever
52

llama-factory

Automate LLaMA-family model fine-tuning via the Llama-Factory WebUI.

Community
Advanced
DonZzzillaDonZzzilla

llama-cpp

Run local GGUF inference for llama.cpp models with Hugging Face Hub discovery.

Community
Advanced
ScientiaCapitalScientiaCapital
1

model-deployment

Export fine-tuned models to GGUF and deploy with vLLM or Ollama.

Community
Advanced