skills.rest
yanochka11
yanochka11COMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Select GGUF models and generate llama.cpp commands for local inference.

Updated May 5, 2026
Search Tags:#model deployment#quantization#gguf#llama.cpp#local inference#hugging face
One-click install
npx skills add https://github.com/yanochka11/harness_bro --skill llama-cpp-yanochka11
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/yanochka11/harness_bro/tree/main/.claude/skills/ported/llama-cpp
Command: npx skills add https://github.com/yanochka11/harness_bro --skill llama-cpp-yanochka11

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps users run local language models efficiently by simplifying GGUF model discovery, quant selection, and llama.cpp inference setup across different hardware environments.

Core Features & Use Cases

  • GGUF Model Discovery: Find llama.cpp-compatible models on Hugging Face and identify available GGUF files and quantization variants.
  • Local Inference Setup: Build llama-cli and llama-server commands for CPU, Apple Silicon, CUDA, ROCm, and Intel GPU environments.
  • Quantization Guidance: Choose suitable Q4, Q5, Q6, IQ, and other quant formats based on memory limits and quality requirements.

Quick Start

Ask the llama-cpp skill to find a suitable GGUF model and create a llama-server command for my hardware.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF models using llama.cpp?▼

Run local GGUF inference by finding compatible Hugging Face models, selecting appropriate quantization formats, and generating llama-server or llama-cli commands tailored to your specific CPU, Apple Silicon, CUDA, ROCm, or Intel GPU hardware configuration.

How do I choose the right GGUF quantization format for my hardware?▼

Choosing the right GGUF quantization format involves evaluating Q4, Q5, Q6, and IQ variants against your system memory limits and quality requirements to ensure the selected quantization balances available RAM with desired inference performance.

Can I deploy llama.cpp for local inference on Apple Silicon and CUDA?▼

Yes, you can deploy llama.cpp for local inference on Apple Silicon and CUDA, as well as CPU, ROCm, and Intel GPU environments, by generating hardware-aware configuration commands tailored to each specific platform's compute capabilities.

What is the best way to find Hugging Face models compatible with llama.cpp?▼

The best way to find compatible Hugging Face models is to use model discovery techniques that identify available GGUF files and quantization variants specifically formatted for llama.cpp local inference deployment workflows.

Does llama.cpp support both command line and server inference modes?▼

Yes, llama.cpp supports both command line and server inference modes by generating specific llama-cli commands for terminal execution and llama-server commands for hosting local model endpoints across various hardware environments.

Related Skills

Explore More Software Engineering Skills→
vikrant-projectvikrant-project
2

llama-cpp

Execute GGUF inference locally with llama.cpp and Python bindings.

Community
Intermediate
ovachieverovachiever
52

llama-factory

Automate LLaMA-family model fine-tuning via the Llama-Factory WebUI.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
juliuss1907juliuss1907

llama-cpp

Run local GGUF inference with llama.cpp models from Hugging Face Hub.

Community
Advanced