skills.rest
Chris-Chai-Minjae
Chris-Chai-MinjaeCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Discover GGUF models on Hugging Face Hub and generate llama-server or llama-cli commands.

Updated Apr 23, 2026
Search Tags:#cli#server#quantization#inference#huggingface#gguf#llama.cpp
One-click install
npx skills add https://github.com/Chris-Chai-Minjae/hermes-agent-r1-bridge --skill llama-cpp-chris-chai-minjae
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/Chris-Chai-Minjae/hermes-agent-r1-bridge/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/Chris-Chai-Minjae/hermes-agent-r1-bridge --skill llama-cpp-chris-chai-minjae

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0.

What problem does it solve?

llama.cpp enables local GGUF model inference and streamlined model discovery for Hugging Face GGUF repositories, removing the need to rely on remote-only options.

Core Features & Use Cases

  • Local GGUF inference across CPU, Apple Silicon, CUDA, ROCm, and Intel GPUs
  • GGUF model discovery and quant selection from Hugging Face Hub
  • Generate server or CLI launch commands (llama-server, llama-cli) for rapid local deployment

Quick Start

Run a quick local llama.cpp workflow by finding a compatible GGUF model on Hugging Face and launching the corresponding llama-server or llama-cli command.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp?▼

Run local GGUF inference by discovering compatible GGUF models on Hugging Face Hub and generating ready-to-run llama-server or llama-cli commands. The Skill maps exact filenames or shorthand quant designations to prepare deployment across your specific hardware.

Can I use llama.cpp for GGUF model discovery on Hugging Face?▼

Yes, GGUF model discovery on Hugging Face works by searching repositories for compatible files and matching exact filenames or shorthand quant designations. It locates suitable models and selects the appropriate quantization before generating deployment commands.

Does local GGUF inference support Apple Silicon and CUDA hardware?▼

Local GGUF inference supports deployment across CPU, Apple Silicon, CUDA, ROCm, and Intel GPUs. The Skill identifies suitable GGUF files and applies the correct hardware-specific configurations to generate executable llama-server or llama-cli commands.

What is the best way to select GGUF quantization for local deployment?▼

Selecting GGUF quantization involves mapping shorthand quant designations or exact filenames from Hugging Face Hub. The Skill evaluates compatible GGUF models and outputs ready-to-run llama-server or llama-cli commands tailored to your chosen quantization and hardware.

Do I need llama-cpp-python to generate llama-server commands?▼

Yes, generating llama-server and llama-cli commands requires the llama-cpp-python dependency, version 0.2.0 or higher. This dependency enables the local GGUF inference workflow by bridging model discovery and command preparation across supported hardware.

Why use GGUF format for local model inference instead of remote APIs?▼

GGUF format enables local model inference without relying on remote-only options, streamlining deployment across various hardware. It allows you to discover compatible files on Hugging Face Hub, select quantization, and launch local llama-server or llama-cli instances.

Related Skills

Explore More Software Engineering Skills→
AlexiosBluffMaraAlexiosBluffMara
2

llama-cpp

Run local GGUF inference and discover models via Hugging Face Hub.

Community
Intermediate
ovachieverovachiever
52

llama-factory

Automate LLaMA-family model fine-tuning via the Llama-Factory WebUI.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
ovachieverovachiever
52

serving-llms-vllm

Deploy OpenAI-compatible LLM APIs with vLLM for high-throughput inference.

Community
Advanced