skills.rest
DifanaDAP
DifanaDAPCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local GGUF inference with llama.cpp and discover Hugging Face Hub models.

Updated Apr 29, 2026
Search Tags:#inference#model discovery#Hugging Face Hub#llama.cpp#CPU Inference
One-click install
npx skills add https://github.com/DifanaDAP/hermes-backup --skill llama-cpp-difanadap
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/DifanaDAP/hermes-backup/tree/main/workspace/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/DifanaDAP/hermes-backup --skill llama-cpp-difanadap

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill streamlines local GGUF inference and model discovery for llama.cpp, reducing the time and complexity of model selection and deployment.

Core Features & Use Cases

  • Local GGUF Inference: Run llama.cpp models on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.
  • Model Discovery: Search and discover Hugging Face Hub models compatible with llama.cpp.
  • Model Building: Build llama-server or llama-cli commands from the Hub.
  • Quick Start: Use the llama skill to run an inference on the Hugging Face Hub model 'bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0'.

Quick Start

To run an inference with the llama skill, use the following command:

llama-cli -hf bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp?▼

Run local GGUF inference by executing `llama-cli -hf bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0` to load and generate text from Hugging Face Hub models directly via llama.cpp.

Can I use llama.cpp for model discovery on the Hugging Face Hub?▼

Yes, llama.cpp model discovery on the Hugging Face Hub allows you to search and find compatible models, then automatically build `llama-server` or `llama-cli` commands for deployment.

Does llama.cpp CPU inference work with Apple Silicon and Intel GPUs?▼

llama.cpp CPU inference supports Apple Silicon, CUDA, ROCm, and Intel GPUs, allowing flexible local model execution across various hardware accelerators without requiring a dedicated server.

Do I need Python and llama-cpp-python to run local GGUF models?▼

Yes, you need Python and the `llama-cpp-python` dependency installed to execute local GGUF inference and perform Hugging Face Hub model discovery tasks within this environment.

What is the best way to find compatible GGUF models for llama.cpp?▼

The best way to find compatible GGUF models is using llama.cpp model discovery to search the Hugging Face Hub, which filters for llama.cpp compatibility and generates ready-to-use `llama-server` commands.

Related Skills

Explore More Software Engineering Skills→
sheawinklersheawinkler
78

llama-cpp

Run local GGUF inference and discover Hugging Face Hub models with llama.cpp.

Community
Advanced
ovachieverovachiever
52

llama-factory

Automate LLaMA-family model fine-tuning via the Llama-Factory WebUI.

Community
Advanced
matthew-johnsonmatthew-johnson

llama-cpp

Run local GGUF inference with llama.cpp and discover Hugging Face Hub models.

Community
Intermediate
SignmanalSignmanal
1

llama-cpp

Run local GGUF model inference with llama.cpp across CPU and GPU hardware.

Community
Intermediate