skills.rest
JuanMS20
JuanMS20COMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local GGUF inference and discover models from Hugging Face Hub.

Updated May 3, 2026
Search Tags:#inference#model-discovery#gguf#local-server#llama.cpp#hugging-face
One-click install
npx skills add https://github.com/JuanMS20/solviora-agent --skill llama-cpp-juanms20
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/JuanMS20/solviora-agent/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/JuanMS20/solviora-agent --skill llama-cpp-juanms20

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0.

What problem does it solve?

llama.cpp local GGUF inference and model discovery for Hugging Face Hub.

Core Features & Use Cases

  • Local GGUF inference on CPU/GPU, including quant selection from HF Hub.
  • HF Hub model discovery and quick deployment with llama-server or llama-cli.
  • Lightweight integration with optional resources for extended tooling.

Quick Start

Install llama.cpp, then locate and load a GGUF model from Hugging Face Hub and run it locally.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using models from Hugging Face Hub?▼

Local GGUF inference is achieved by discovering a compatible model on Hugging Face Hub, selecting the appropriate quantization, and loading it locally via llama-server or llama-cli for immediate generation.

What is the best way to select a quantized GGUF model for local CPU or GPU inference?▼

Quant selection for GGUF models involves evaluating Hugging Face Hub variants to balance memory footprint and inference speed for your local CPU or GPU hardware configuration.

Can I use llama-cli and llama-server for quick local model deployment?▼

Yes, both llama-cli and llama-server support quick deployment by loading a specified GGUF model locally, enabling immediate inference without extensive configuration overhead.

How does llama.cpp integration handle Hugging Face model discovery?▼

Hugging Face model discovery is handled by searching the Hub for compatible GGUF files, allowing users to locate, evaluate, and download specific quantized models for local execution.

What are the limitations of running local GGUF inference with this setup?▼

Limitations depend on local hardware constraints, as CPU or GPU memory capacity dictates the maximum model size and quantization level available for GGUF inference.

Related Skills

Explore More Software Engineering Skills→
samuelmukotisamuelmukoti

llama-cpp

Run local LLM inference with llama.cpp and GGUF models on CPU or GPU.

Community
Advanced
x-TheFoxx-TheFox

llama-cpp

Run local GGUF inference across CPU, Apple Silicon, and GPU backends.

Community
Intermediate
eliottbusinesseliottbusiness

llama-cpp

Run llama.cpp GGUF model inference locally with Hugging Face Hub discovery.

Community
Advanced
Monjyu1101Monjyu1101
1

llama-cpp

Configure local GGUF inference and discover Hugging Face models for llama.cpp.

Community
Intermediate