llama-cpp

Discover GGUF models on Hugging Face and launch llama.cpp inference commands.

Updated Feb 21, 2026

Search Tags:#quantization #gguf #llama.cpp #llm deployment #local inference #hugging face

Ready to run

npx skills add https://github.com/Gitnapp/Skills --skill llama-cpp-gitnapp

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/Gitnapp/Skills/tree/main/mlops/inference/llama-cpp
Command: npx skills add https://github.com/Gitnapp/Skills --skill llama-cpp-gitnapp

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps users run and manage local llama.cpp inference workflows by simplifying GGUF model discovery, quant selection, and deployment across different hardware environments.

Core Features & Use Cases

GGUF Model Discovery: Find llama.cpp-compatible models on Hugging Face, inspect available GGUF files, and identify suitable quantizations.
Local Inference Setup: Build llama-cli and llama-server commands for CPU, Apple Silicon, CUDA, ROCm, and Intel GPU environments.
Use Case: Configure a local AI assistant by selecting an appropriate GGUF quantization for available RAM or VRAM and launching an OpenAI-compatible llama.cpp server.

Quick Start

Use the llama-cpp skill to find the best GGUF model and generate a llama-server command for my hardware.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run local GGUF models with llama.cpp on my hardware?▼

Run local GGUF models by generating optimized llama-cli or llama-server commands tailored to your CPU, Apple Silicon, CUDA, or ROCm environment. The Skill configures deployment based on available hardware to launch local inference workflows.

What is the best way to find compatible GGUF models on Hugging Face for local inference?▼

Finding compatible GGUF models involves searching Hugging Face repositories and inspecting available GGUF files. The Skill identifies suitable quantizations to ensure compatibility with your local llama.cpp deployment and specific hardware constraints.

How do I choose the right GGUF quantization for my available RAM or VRAM?▼

Choosing the right GGUF quantization requires evaluating your available RAM or VRAM against model requirements. The Skill simplifies quantization selection to match your hardware capacity for stable local inference.

Can I use llama.cpp to set up an OpenAI-compatible local server?▼

You can use llama.cpp to set up an OpenAI-compatible local server by generating the appropriate llama-server commands. The Skill configures production-style local serving workflows for your specific CPU or GPU environment.

Does llama-cpp-python integration support GPU acceleration across different platforms?▼

llama-cpp-python integration supports GPU acceleration across CUDA, ROCm, and Intel GPU environments. The Skill applies these integration patterns to build optimized inference commands for your specific hardware configuration.

Why should I use GGUF quantization for local LLM deployment instead of other formats?▼

GGUF quantization enables efficient local LLM deployment by reducing model size to fit within hardware memory limits. The Skill simplifies discovering and selecting these quantized models for optimized llama.cpp inference.