skills.rest
DonZzzilla
DonZzzillaCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

llama-cpp

Run local GGUF inference for llama.cpp models with Hugging Face Hub discovery.

Updated Jun 13, 2026
Search Tags:#GGUF#Hugging Face Hub#llama.cpp#local deployment#model inference
One-click install
npx skills add https://github.com/DonZzzilla/kalshi-explorer --skill llama-cpp-donzzzilla
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/DonZzzilla/kalshi-explorer/tree/main/skills/user-skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/DonZzzilla/kalshi-explorer --skill llama-cpp-donzzzilla

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill simplifies local GGUF inference and Hugging Face Hub model discovery for llama.cpp, enabling efficient model selection and deployment.

Core Features & Use Cases

  • Local GGUF Inference: Run llama.cpp models locally on CPU, GPU, or Apple Silicon.
  • Model Discovery: Find and select the right GGUF model for a specific Hugging Face repository.
  • Deployment: Build and run llama-server or llama-cli commands from the Hub.
  • Use Case: You have a specific Hugging Face model in mind and want to quickly find the best quantized version and run it locally for your application.

Quick Start

To use the llama.cpp skill for model discovery, run:

llama-cli -hf <repo>:<QUANT>

Replace <repo> with the Hugging Face model repository name and <QUANT> with the desired quantization level (e.g., Q8_0).

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp on different hardware?▼

Local GGUF inference is supported across CPU, Apple Silicon, CUDA, ROCm, and Intel GPU hardware. This Skill configures the llama.cpp library and Python bindings to execute models directly on your machine. It handles hardware-specific deployment automatically.

How do I find and select a GGUF model from the Hugging Face Hub?▼

You can discover GGUF models on the Hugging Face Hub by specifying the repository and desired quantization level. The Skill facilitates model discovery by helping you find the right quantized version for your specific application needs. It builds commands to fetch files directly.

Can I convert Transformers weights to GGUF format for local deployment?▼

Yes, converting Transformers weights to GGUF format is supported for local deployment. This allows you to take existing Hugging Face models and prepare them for efficient inference. The Skill manages the conversion and subsequent file selection process.

What is the best way to start a llama-server or llama-cli command from the Hub?▼

The best way to start is using the `llama-cli -hf <repo>:<QUANT>` command. Replace `<repo>` with the Hugging Face repository name and `<QUANT>` with the desired quantization level. This builds and runs the server or CLI directly from the Hub.

Does this local deployment approach work with both llama-server and llama-cli?▼

Yes, local deployment works with both llama-server and llama-cli. The Skill builds and generates the necessary commands for either interface from the Hugging Face Hub. You can choose the best interface for your specific inference application.

Do I need the llama.cpp Python bindings to use this Skill for model discovery?▼

Yes, the llama.cpp library and Python bindings are required dependencies. You need llama-cpp-python version 0.2.0 or higher installed. This provides the underlying engine for local GGUF inference and Hugging Face Hub integration.

Related Skills

Explore More Software Engineering Skills→
AlexiosBluffMaraAlexiosBluffMara
2

llama-cpp

Run local GGUF inference and discover models via Hugging Face Hub.

Community
Intermediate
x-TheFoxx-TheFox

llama-cpp

Run local GGUF inference across CPU, Apple Silicon, and GPU backends.

Community
Intermediate
DarkArty07DarkArty07
3

llama-cpp

Deploy local LLM inference with GGUF quantized weights via llama.cpp.

Community
Advanced
gztcodegztcode

llama-cpp

Run large language model inference across CPU, Apple Silicon, and GPU platforms.

Community
Advanced