llama-cpp

Name: llama-cpp
Availability: InStock
Author: yanochka11

Select GGUF models and generate llama.cpp commands for local inference.

Updated May 5, 2026

Search Tags:#model deployment #quantization #gguf #llama.cpp #local inference #hugging face

Ready to run

npx skills add https://github.com/yanochka11/harness_bro --skill llama-cpp-yanochka11

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/yanochka11/harness_bro/tree/main/.claude/skills/ported/llama-cpp
Command: npx skills add https://github.com/yanochka11/harness_bro --skill llama-cpp-yanochka11

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps users run local language models efficiently by simplifying GGUF model discovery, quant selection, and llama.cpp inference setup across different hardware environments.

Core Features & Use Cases

GGUF Model Discovery: Find llama.cpp-compatible models on Hugging Face and identify available GGUF files and quantization variants.
Local Inference Setup: Build llama-cli and llama-server commands for CPU, Apple Silicon, CUDA, ROCm, and Intel GPU environments.
Quantization Guidance: Choose suitable Q4, Q5, Q6, IQ, and other quant formats based on memory limits and quality requirements.

Quick Start

Ask the llama-cpp skill to find a suitable GGUF model and create a llama-server command for my hardware.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run local GGUF models using llama.cpp?▼

Run local GGUF inference by finding compatible Hugging Face models, selecting appropriate quantization formats, and generating llama-server or llama-cli commands tailored to your specific CPU, Apple Silicon, CUDA, ROCm, or Intel GPU hardware configuration.

How do I choose the right GGUF quantization format for my hardware?▼

Choosing the right GGUF quantization format involves evaluating Q4, Q5, Q6, and IQ variants against your system memory limits and quality requirements to ensure the selected quantization balances available RAM with desired inference performance.

Can I deploy llama.cpp for local inference on Apple Silicon and CUDA?▼

Yes, you can deploy llama.cpp for local inference on Apple Silicon and CUDA, as well as CPU, ROCm, and Intel GPU environments, by generating hardware-aware configuration commands tailored to each specific platform's compute capabilities.

What is the best way to find Hugging Face models compatible with llama.cpp?▼

The best way to find compatible Hugging Face models is to use model discovery techniques that identify available GGUF files and quantization variants specifically formatted for llama.cpp local inference deployment workflows.

Does llama.cpp support both command line and server inference modes?▼

Yes, llama.cpp supports both command line and server inference modes by generating specific llama-cli commands for terminal execution and llama-server commands for hosting local model endpoints across various hardware environments.