skills.rest
Piaoxuemoli
PiaoxuemoliCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local GGUF inference with llama.cpp models from Hugging Face Hub.

Updated Jun 15, 2026
Search Tags:#GGUF#quantization#Hugging Face Hub#llama.cpp#model inference
One-click install
npx skills add https://github.com/Piaoxuemoli/qoobeeHermes --skill llama-cpp-piaoxuemoli
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/Piaoxuemoli/qoobeeHermes/tree/main/hermes/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/Piaoxuemoli/qoobeeHermes --skill llama-cpp-piaoxuemoli

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill streamlines local GGUF inference and model discovery for llama.cpp, enabling users to run models on various hardware and find the right quantization for their needs.

Core Features & Use Cases

  • Local GGUF Inference: Run llama.cpp models on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.
  • Model Discovery: Find the right GGUF for a specific Hugging Face repo, including size constraints and variant selection.
  • Command Generation: Build llama-server or llama-cli commands directly from the Hub.
  • Use Case: Suppose you want to use a llama.cpp model for a chatbot on a Raspberry Pi. Use this Skill to find a lightweight model, quantize it, and generate the command to run it on your device.

Quick Start

Use the llama-cpp skill to run the model 'bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0' on your local machine.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference on edge devices like a Raspberry Pi?▼

You can perform local GGUF inference on edge devices by finding a lightweight model from Hugging Face Hub, selecting a suitable quantization, and generating the llama-cli command to execute it on your hardware.

How do I find the right quantized GGUF model for my hardware on Hugging Face Hub?▼

Finding the right GGUF model involves using model discovery features to search a specific Hugging Face repo, filter by size constraints, and select the best quantization variant for your CPU, GPU, or edge device scenario.

Can I run llama.cpp models on Apple Silicon and CUDA GPUs?▼

Yes, local GGUF inference supports various hardware configurations including Apple Silicon, CUDA, ROCm, and Intel GPUs, allowing you to run llama.cpp models across these platforms.

Do I need llama-cpp-python installed to generate llama-server commands?▼

Yes, llama-cpp-python version 0.2.0 or higher is required to perform local GGUF model inference and generate llama-server or llama-cli commands from Hugging Face Hub.

What is the best way to build llama-server commands from a Hugging Face repo?▼

The best way to build llama-server commands is to use command generation features that construct execution syntax directly from your selected GGUF model and Hugging Face Hub repository.

Related Skills

Explore More Software Engineering Skills→
sheawinklersheawinkler
78

llama-cpp

Run local GGUF inference and discover Hugging Face Hub models with llama.cpp.

Community
Advanced
x-TheFoxx-TheFox

llama-cpp

Run local GGUF inference across CPU, Apple Silicon, and GPU backends.

Community
Intermediate
ldzhhxxldzhhxx
1

llama-cpp

Run Hugging Face Hub GGUF models locally via llama.cpp on CPU or GPU.

Community
Advanced
juliuss1907juliuss1907

llama-cpp

Run local GGUF inference with llama.cpp models from Hugging Face Hub.

Community
Advanced