skills.rest
samuelbonifacio015
samuelbonifacio015COMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local GGUF inference with llama.cpp models from Hugging Face Hub.

Updated Aug 23, 2026
Search Tags:#inference#gguf#llama.cpp#cpu inference#hugging face
One-click install
npx skills add https://github.com/samuelbonifacio015/obsidian-notes --skill llama-cpp-samuelbonifacio015
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/samuelbonifacio015/obsidian-notes/tree/main/obsidian/Sistema/Skills/Vendor/mlops/inference/llama-cpp
Command: npx skills add https://github.com/samuelbonifacio015/obsidian-notes --skill llama-cpp-samuelbonifacio015

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill facilitates local GGUF inference and provides discovery of llama.cpp models from the Hugging Face Hub, eliminating the need for external dependencies.

Core Features & Use Cases

  • Local GGUF Inference: Perform inference directly on your local system with llama.cpp, supporting various hardware configurations.
  • Model Discovery: Search and discover llama.cpp models from the Hugging Face Hub with specific quantization options.
  • Use Case: If you need to run a model like 'bartowski/Llama-3.2-3B-Instruct-GGUF' with Q8_0 quantization on your local machine, this Skill can help you find the correct GGUF file and execute it.

Quick Start

Run the model 'bartowski/Llama-3.2-3B-Instruct-GGUF' with Q8_0 quantization directly from the Hugging Face Hub using the 'llama-server' command:

llama-server -hf bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using a model from Hugging Face Hub?▼

Local GGUF inference from Hugging Face Hub can be executed directly using the llama-server command, specifying the model repository and desired quantization like Q8_0. This allows hardware-specific local inference without external dependencies.

Can I search for llama.cpp models with specific quantization on Hugging Face?▼

Yes, you can discover and search for llama.cpp models from the Hugging Face Hub with specific quantization options. This model discovery feature helps you find the correct GGUF file for your hardware configuration.

Do I need llama.cpp installed to run Hugging Face GGUF models locally?▼

Yes, local GGUF inference requires llama.cpp and Hugging Face Hub API access. The Skill uses the llama-cpp-python dependency to perform inference directly on your local system across various hardware configurations.

What is the best way to execute a specific GGUF quantization like Q8_0 locally?▼

The best way to execute GGUF quantization locally is using the llama-server command with the Hugging Face Hub repository and quantization tag, such as llama-server -hf bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0.

Does local llama.cpp inference support different hardware configurations?▼

Local llama.cpp inference supports various hardware-specific configurations for running GGUF models. This allows you to perform inference directly on your local system optimized for your specific platform setup.

Related Skills

Explore More Software Engineering Skills→
sheawinklersheawinkler
78

llama-cpp

Run local GGUF inference and discover Hugging Face Hub models with llama.cpp.

Community
Advanced
ashiqcodeleafashiqcodeleaf

llama-cpp

Run local GGUF inference with llama.cpp and discover Hugging Face Hub models.

Community
Advanced
eliottbusinesseliottbusiness

llama-cpp

Run llama.cpp GGUF model inference locally with Hugging Face Hub discovery.

Community
Advanced
aayushsoamaayushsoam
1

llama-cpp

Run llama.cpp GGUF model inference locally across CPU, Apple Silicon, CUDA, ROCm, and Intel GPUs.

Community
Advanced