skills.rest
maystudios
maystudiosCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

llama-cpp

Build, configure, and deploy llama.cpp for local LLM inference.

15|1|Updated Apr 16, 2026
Search Tags:#cmake#quantization#llama-cpp#c-api#gguf#unreal-engine#gpu-backends
One-click install
npx skills add https://github.com/maystudios/claude-skills --skill llama-cpp-maystudios
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/maystudios/claude-skills/tree/main/llama-cpp
Command: npx skills add https://github.com/maystudios/claude-skills --skill llama-cpp-maystudios

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Simplifies the complex, multi-step process of building, quantizing, and deploying llama.cpp so teams can run high-performance local LLMs without guesswork. It consolidates C/C++ integration patterns, GPU backend builds, model conversion, server deployment, and engine-specific embedding guidance into a single practical reference.

Core Features & Use Cases

  • C API & C++ wrappers: Complete reference for llama.h and llama-cpp.h usage, context creation, tokenization, batching, and sampler workflows for embedding the library directly into applications.
  • Builds & GPU backends: Detailed CMake options and recipes for CUDA, Vulkan, Metal, HIP/ROCm, SYCL, and dynamic backend loading to maximize performance on diverse hardware.
  • GGUF & Quantization: End-to-end instructions for converting HuggingFace models to GGUF, quantizing (Q4_K_M, Q8_0, IQ4_XS, etc.), and generating importance matrices for IQ quant types.
  • Server & API: How to run the built-in OpenAI-compatible llama-server, use its endpoints (chat, completions, embeddings), enable streaming, function calling, grammar/JSON constraints, and speculative decoding.
  • Integration examples: CMake add_subdirectory and find_package patterns, Docker images, static-library builds for plugin systems, and an extensive Unreal Engine integration guide for in-game inference.

Quick Start

Start the built-in llama-server with a GGUF model, enable the appropriate GPU backend, and query the OpenAI-compatible /v1/chat/completions endpoint to verify local inference.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build llama.cpp with CUDA, Vulkan, or Metal GPU backends?▼

Build llama.cpp with GPU backends by setting specific CMake options for CUDA, Vulkan, Metal, or HIP/ROCm to optimize local LLM inference performance across diverse hardware environments.

How do I convert HuggingFace models to GGUF and quantize them for local inference?▼

Convert HuggingFace models to GGUF format and apply quantization methods like Q4_K_M, Q8_0, or IQ4_XS to reduce model size and enable efficient local execution.

Can I integrate the llama.cpp C API directly into my C++ application?▼

Integrate llama.cpp directly using the llama.h and llama-cpp.h C API wrappers for context creation, tokenization, batching, and sampler workflows within your application.

Does llama.cpp provide an OpenAI-compatible server API for local deployment?▼

Run the built-in llama-server to expose an OpenAI-compatible API, enabling streaming, function calling, grammar constraints, and speculative decoding for local inference.

What is the best way to run local LLM inference inside Unreal Engine?▼

Embed local LLM inference in Unreal Engine by following the extensive integration guide, utilizing CMake add_subdirectory patterns and static-library builds for in-game deployment.

Do I need CMake to use llama.cpp for local LLM deployment?▼

CMake is required to build llama.cpp, configure GPU backend flags for hardware acceleration, and manage project integration patterns like add_subdirectory and find_package.

Related Skills

Explore More Software Engineering Skills→
founderphantomfounderphantom

llama-cpp

Perform CPU-based LLM inference with GGUF quantization using llama.cpp.

Community
Intermediate
kaminocorpkaminocorp
228

llama-cpp

Run C/C++ LLM inference on CPU and non-NVIDIA hardware with GGUF quantization.

Official
Intermediate
martimramosmartimramos
1

forge-lang-cpp

Standardize C++ project builds, tests, and static analysis with CMake and Make.

Community
Intermediate
1thirteeng31thirteeng3

llama-cpp

Run LLM inference on CPUs and non-NVIDIA hardware with GGUF quantization.

Community
Intermediate