skills.rest
kngender5
kngender5COMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Discover Hugging Face Hub models and build llama.cpp inference command lines.

Updated Jun 4, 2026
Search Tags:#GGUF#inference#model discovery#llama.cpp#command line building
One-click install
npx skills add https://github.com/kngender5/hermes --skill llama-cpp-kngender5
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/kngender5/hermes/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/kngender5/hermes --skill llama-cpp-kngender5

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill provides a comprehensive solution for local GGUF inference with llama.cpp, allowing users to discover models from the Hugging Face Hub and build custom command lines for local inference.

Core Features & Use Cases

  • Local Inference: Run llama.cpp models locally on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.
  • Model Discovery: Search and select models from the Hugging Face Hub based on specific criteria such as size constraints or compatibility with llama.cpp.
  • Command Line Building: Generate command lines for llama-server or llama-cli based on model and quant selection.
  • Use Case: If you need to perform fast inference with a small model that doesn't require GPU acceleration, you can use this Skill to find a Q2_K GGUF file from a specific Hugging Face Hub repository and build a command line for llama-cli.

Quick Start

To use a specific model from the Hugging Face Hub for inference with llama-cli, run the following command:

llama-cli -hf <model-repo>:<quant-label>

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp without a dedicated GPU?▼

You can run local GGUF inference on CPU by building a llama-cli command line, selecting a small quantized model like Q2_K from the Hugging Face Hub for fast local inference without GPU acceleration.

What's the best way to discover GGUF models on the Hugging Face Hub for llama.cpp?▼

Discover GGUF models on the Hugging Face Hub by searching based on specific criteria such as size constraints or llama.cpp compatibility, then build a custom command line for the selected model and quantization.

How do I build a command line for llama-server or llama-cli from a specific model?▼

Build a command line for llama-server or llama-cli by specifying the model repository and quantization label, using the quick start format: llama-cli -hf <model-repo>:<quant-label> for direct inference.

Does llama.cpp inference work on Apple Silicon and CUDA environments?▼

Local llama.cpp inference works across multiple hardware backends including CPU, Apple Silicon, CUDA, ROCm, and Intel GPUs, allowing flexible deployment in various AI research and testing environments.

Do I need to install llama.cpp before running local GGUF models?▼

Yes, you must have llama.cpp installed and configured locally, and the llama-cpp-python dependency must be present to perform local GGUF inference, model discovery, and command line building.

Related Skills

Explore More Software Engineering Skills→
x-TheFoxx-TheFox

llama-cpp

Run local GGUF inference across CPU, Apple Silicon, and GPU backends.

Community
Intermediate
BermudaLocalsBermudaLocals
1

llama-cpp

Execute local GGUF inference and discover Hugging Face models with llama.cpp.

Community
Intermediate
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
helix4uhelix4u

llama-cpp

Run LLM inference on CPU, Apple Silicon, and non-NVIDIA GPUs with GGUF models.

Community
Intermediate