llama-cpp

Name: llama-cpp
Availability: InStock
Author: mark

Performs GGUF inference and discovers Hugging Face Hub models for llama.cpp across CPU and GPU backends.

31|3|Updated May 7, 2026

Search Tags:#gpu #inference #cpu #llama-cpp #model-discovery #gguf #hugging-face

Ready to run

npx skills add https://github.com/markwang2658/hermes-windows-native --skill llama-cpp-markwang2658

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/markwang2658/hermes-windows-native/tree/main/hermes-agent/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/markwang2658/hermes-windows-native --skill llama-cpp-markwang2658

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes references (resource) components.

What problem does it solve?

Enables local GGUF inference with llama.cpp and simplifies Hugging Face Hub-based model discovery, removing the need for Docker or WSL2.

Core Features & Use Cases

Local GGUF inference on CPU, Apple Silicon, or GPU backends via llama-cpp-python.
Quantization guidance and model discovery directly from Hugging Face Hub.
HF local-app and tree API workflows enable quick selection of compatible GGUF files.

Quick Start

Run the llama.cpp Quick Start to perform local GGUF inference and model discovery.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run local GGUF inference without using Docker or WSL2?▼

You can run local GGUF inference directly on Linux, macOS, or Windows using llama-cpp-python, which removes the need for Docker or WSL2 by selecting native CPU or GPU backends.

How do I find compatible GGUF models on Hugging Face Hub?▼

Find compatible GGUF models on Hugging Face Hub using model discovery workflows that query HF local-app and tree APIs, enabling quick selection of files that match your hardware backend.

Can I use Apple Silicon GPU acceleration for local llama.cpp inference?▼

Yes, local llama.cpp inference supports Apple Silicon GPU acceleration alongside standard CPU and GPU backends across Linux, macOS, and Windows environments.

Do I need a specific version of llama-cpp-python to run GGUF models locally?▼

Yes, you need llama-cpp-python version 0.2.0 or higher because the skill enforces this dependency compatibility to ensure proper quantization selection and local server execution.

What is the best way to get quantization guidance for local GGUF models?▼

The best way to get quantization guidance for local GGUF models is using the skill's built-in discovery features, which match Hugging Face GGUF files to your specific CPU or GPU backend.