skills.rest
Gitnapp
GitnappCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Discover GGUF models on Hugging Face and launch llama.cpp inference commands.

Updated Feb 21, 2026
Search Tags:#quantization#gguf#llama.cpp#llm deployment#local inference#hugging face
One-click install
npx skills add https://github.com/Gitnapp/Skills --skill llama-cpp-gitnapp
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/Gitnapp/Skills/tree/main/mlops/inference/llama-cpp
Command: npx skills add https://github.com/Gitnapp/Skills --skill llama-cpp-gitnapp

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps users run and manage local llama.cpp inference workflows by simplifying GGUF model discovery, quant selection, and deployment across different hardware environments.

Core Features & Use Cases

  • GGUF Model Discovery: Find llama.cpp-compatible models on Hugging Face, inspect available GGUF files, and identify suitable quantizations.
  • Local Inference Setup: Build llama-cli and llama-server commands for CPU, Apple Silicon, CUDA, ROCm, and Intel GPU environments.
  • Use Case: Configure a local AI assistant by selecting an appropriate GGUF quantization for available RAM or VRAM and launching an OpenAI-compatible llama.cpp server.

Quick Start

Use the llama-cpp skill to find the best GGUF model and generate a llama-server command for my hardware.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF models with llama.cpp on my hardware?▼

Run local GGUF models by generating optimized llama-cli or llama-server commands tailored to your CPU, Apple Silicon, CUDA, or ROCm environment. The Skill configures deployment based on available hardware to launch local inference workflows.

What is the best way to find compatible GGUF models on Hugging Face for local inference?▼

Finding compatible GGUF models involves searching Hugging Face repositories and inspecting available GGUF files. The Skill identifies suitable quantizations to ensure compatibility with your local llama.cpp deployment and specific hardware constraints.

How do I choose the right GGUF quantization for my available RAM or VRAM?▼

Choosing the right GGUF quantization requires evaluating your available RAM or VRAM against model requirements. The Skill simplifies quantization selection to match your hardware capacity for stable local inference.

Can I use llama.cpp to set up an OpenAI-compatible local server?▼

You can use llama.cpp to set up an OpenAI-compatible local server by generating the appropriate llama-server commands. The Skill configures production-style local serving workflows for your specific CPU or GPU environment.

Does llama-cpp-python integration support GPU acceleration across different platforms?▼

llama-cpp-python integration supports GPU acceleration across CUDA, ROCm, and Intel GPU environments. The Skill applies these integration patterns to build optimized inference commands for your specific hardware configuration.

Why should I use GGUF quantization for local LLM deployment instead of other formats?▼

GGUF quantization enables efficient local LLM deployment by reducing model size to fit within hardware memory limits. The Skill simplifies discovering and selecting these quantized models for optimized llama.cpp inference.

Related Skills

Explore More Software Engineering Skills→
richardnguyen0715richardnguyen0715

llama-cpp

Execute local GGUF inference for llama.cpp models across multiple hardware platforms.

Community
Advanced
jayll1303jayll1303
17

llama-cpp-inference

Run local GGUF model inference and OpenAI-compatible serving with llama.cpp.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
helix4uhelix4u

llama-cpp

Run LLM inference on CPU, Apple Silicon, and non-NVIDIA GPUs with GGUF models.

Community
Intermediate