Verified:GitHub

llama-cpp

Run local GGUF inference with llama.cpp models from the Hugging Face Hub.

1|Updated Jun 9, 2026

Search Tags:#GGUF #model discovery #Hugging Face #llama.cpp #local inference

One-click install

npx skills add https://github.com/aivos-xie/hermes-skills --skill llama-cpp-aivos-xie

Or copy as Structured Prompt for Agent▼

Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/aivos-xie/hermes-skills/tree/main/mlops/inference/llama-cpp
Command: npx skills add https://github.com/aivos-xie/hermes-skills --skill llama-cpp-aivos-xie

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill facilitates local GGUF inference and streamlined discovery of models from the Hugging Face Hub, optimizing performance and customization for llama.cpp users.

Core Features & Use Cases

Local GGUF Inference: Execute llama.cpp models locally on various hardware platforms.
Model Discovery: Search, select, and launch Hugging Face Hub models for GGUF support.
Quick Deployment: Build and run llama-server or llama-cli commands directly from the Hub.
Use Case: For a user developing AI applications and seeking to utilize llama.cpp models, this Skill provides a convenient way to try different models locally before deploying them in a production environment.

Quick Start

Run the model from the Hugging Face Hub using:

llama-server -hf bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

What's the best way to search for and discover GGUF models on the Hugging Face Hub?▼

Do I need llama-cpp-python installed to run local GGUF inference?

Do I need llama-cpp-python installed to run local GGUF inference?▼

Can I build and run llama-cli commands directly from Hugging Face models?

Can I build and run llama-cli commands directly from Hugging Face models?▼

When should I use llama.cpp for local inference instead of cloud deployment?

When should I use llama.cpp for local inference instead of cloud deployment?▼

Use llama.cpp for local GGUF inference when you need to optimize performance and customization across various hardware platforms before deploying applications to production.