skills.rest
richardnguyen0715
richardnguyen0715COMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Execute local GGUF inference for llama.cpp models across multiple hardware platforms.

Updated May 11, 2026
Search Tags:#GGUF#Hugging Face Hub#llama.cpp#local execution#model inference
One-click install
npx skills add https://github.com/richardnguyen0715/keep-it-real --skill llama-cpp-richardnguyen0715
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/richardnguyen0715/keep-it-real/tree/main/refer-projects/hermes-agent/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/richardnguyen0715/keep-it-real --skill llama-cpp-richardnguyen0715

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill allows for local inference with llama.cpp, providing efficient and versatile access to quantized models for a variety of hardware and use cases.

Core Features & Use Cases

  • Local GGUF Inference: Run llama.cpp models on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.
  • Model Discovery: Find and select the appropriate Hugging Face Hub models for llama.cpp.
  • Command-Line Tools: Access llama-server and llama-cli for model inference and management.
  • Use Case: Suppose you are developing a text generation tool and need to quickly find and test a llama.cpp model from the Hugging Face Hub for your application.

Quick Start

Run the llama.cpp model from the Hugging Face Hub with the command: llama-server -hf bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp models from Hugging Face Hub?▼

Run local GGUF inference by using the llama-server command with a Hugging Face Hub model repository and quantization tag, such as: llama-server -hf bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0.

Can I execute llama.cpp inference on Apple Silicon and CUDA hardware?▼

Yes, local GGUF inference supports multiple hardware configurations including CPU, Apple Silicon, CUDA, ROCm, and Intel GPUs, allowing efficient execution of quantized models across diverse hardware setups.

What is the best way to discover and select Hugging Face Hub models for llama.cpp?▼

Model discovery is handled through a comprehensive interface that helps find and select appropriate Hugging Face Hub models for llama.cpp, suitable for AI research, development, and production environments.

Do I need llama-cpp-python installed to run local GGUF models?▼

Yes, executing local GGUF inference requires the llama-cpp-python bindings version 0.2.0 or higher, along with the actual GGUF model files to perform text generation and inference tasks.

Are there limitations when running quantized models locally with llama.cpp?▼

Local GGUF inference requires specific hardware configurations and GGUF formatted model files. You must ensure your system matches supported platforms like CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs for optimal quantized model performance.

Related Skills

Explore More Software Engineering Skills→
AlexiosBluffMaraAlexiosBluffMara
2

llama-cpp

Run local GGUF inference and discover models via Hugging Face Hub.

Community
Intermediate
Simon-Copilot-StudioSimon-Copilot-Studio

unsloth

Configures LoRA/QLoRA fine-tuning workflows for LLMs with Unsloth on local, Colab, or Docker environments.

Community
Intermediate
eliottbusinesseliottbusiness

llama-cpp

Run llama.cpp GGUF model inference locally with Hugging Face Hub discovery.

Community
Advanced
vikrant-projectvikrant-project
2

llama-cpp

Execute GGUF inference locally with llama.cpp and Python bindings.

Community
Intermediate