skills.rest
Toqsick
ToqsickCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local GGUF inference for llama.cpp models with quantization selection and Hugging Face Hub discovery.

Updated Jul 3, 2026
Search Tags:#model discovery#Hugging Face Hub#llama.cpp#local execution#GGUF inference
One-click install
npx skills add https://github.com/Toqsick/MaxClaw --skill llama-cpp-toqsick
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/Toqsick/MaxClaw/tree/main/.claude/skills/llama-cpp
Command: npx skills add https://github.com/Toqsick/MaxClaw --skill llama-cpp-toqsick

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill simplifies local GGUF inference and Hugging Face model discovery for llama.cpp, streamlining the process of finding, using, and evaluating llama.cpp models.

Core Features & Use Cases

  • GGUF Inference: Locally execute llama.cpp models with GGUF for inference and quant selection.
  • Model Discovery: Discover and select models from the Hugging Face Hub that support llama.cpp.
  • Use Case: When you need to run a llama.cpp model locally, this Skill helps you find the right model, download the GGUF file, and run it with the appropriate command.

Quick Start

Run the llama.cpp model with GGUF 'Q4_K_M' from the Hugging Face Hub: llama-server -hf bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_M

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using a model from the Hugging Face Hub?▼

Local GGUF inference is enabled by discovering a llama.cpp model on Hugging Face Hub, selecting a quantization format like Q4_K_M, and executing it locally via the llama-server command.

What is the best way to discover Hugging Face models that support llama.cpp?▼

Hugging Face model discovery for llama.cpp is handled by the Skill's built-in search functionality, which identifies compatible GGUF files and allows you to select the appropriate quantization for local execution.

Do I need llama-cpp-python installed to run GGUF models locally?▼

Yes, the llama-cpp-python package version 0.2.0 or higher is required as a dependency to enable the local execution of GGUF models and facilitate inference through the llama.cpp framework.

Can I select different quantization formats when downloading GGUF files?▼

Yes, quantization selection is supported during model discovery, allowing you to specify formats like Q4_K_M directly in the execution command to optimize local GGUF inference performance and memory usage.

Does GGUF inference work across different operating system platforms?▼

Local GGUF inference is optimized to support various platforms, enabling consistent execution of llama.cpp models across different operating environments using the required Python bindings.

What are the limitations of using llama.cpp for local model execution?▼

Local GGUF inference performance depends entirely on your hardware capabilities and the selected quantization format, requiring sufficient memory to load the model and execute the llama.cpp framework efficiently.

Related Skills

Explore More Software Engineering Skills→
richardnguyen0715richardnguyen0715

llama-cpp

Execute local GGUF inference for llama.cpp models across multiple hardware platforms.

Community
Advanced
SignmanalSignmanal
1

llama-cpp

Run local GGUF model inference with llama.cpp across CPU and GPU hardware.

Community
Intermediate
DonZzzillaDonZzzilla

llama-cpp

Run local GGUF inference for llama.cpp models with Hugging Face Hub discovery.

Community
Advanced
mahg-esmahg-es
3

llm-local-deploy

Deploy local LLMs with llama.cpp, Ollama, or vLLM via docker-compose.

Official
Intermediate