skills.rest
magnus919
magnus919COMMUNITY
Explore all skills by this author →
WebsiteGitHub

llama-cpp

Operate and benchmark llama.cpp across CPU, CUDA, and Metal backends.

40|6|Updated Jul 11, 2026
Search Tags:#llm#benchmarking#inference#cuda#serving#gguf#llama.cpp
One-click install
npx skills add https://github.com/magnus919/agent-skills --skill llama-cpp-magnus919
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/magnus919/agent-skills/tree/main/llama-cpp
Command: npx skills add https://github.com/magnus919/agent-skills --skill llama-cpp-magnus919

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) and templates (resource) and evals (resource) components.

What problem does it solve?

This skill eliminates the guesswork in running local LLMs by providing a discovery-first operating procedure that verifies hardware, model compatibility, and backend performance before execution.

Core Features & Use Cases

  • Hardware-Aware Configuration: Automatically detects and configures CPU, Metal, CUDA, HIP, and Vulkan backends for optimal performance.
  • Verified Inference & Serving: Provides structured workflows for running CLI inference and OpenAI-compatible API servers with robust health checks.
  • Performance Benchmarking: Enables reproducible tuning of context, batching, and offloading to ensure your local models meet latency and throughput requirements.

Quick Start

Use the llama-cpp skill to inspect your hardware and run a local inference test on the model file located at /models/chat.gguf.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local LLM inference using a GGUF model file?▼

Local LLM inference with a GGUF model file is executed through structured workflows that verify hardware compatibility and configure CPU, CUDA, or Metal backends before running CLI inference.

What is the best way to benchmark llama.cpp performance across different hardware backends?▼

Benchmarking llama.cpp involves reproducible tuning of context, batching, and accelerator offloading to ensure local models meet specific latency and throughput requirements across CPU, CUDA, and Metal.

Can I deploy an OpenAI-compatible API server using llama.cpp?▼

Yes, you can deploy an OpenAI-compatible API server using llama.cpp through structured workflows that include robust health checks to ensure serving stability.

Does llama.cpp support automatic hardware detection for CUDA and Metal acceleration?▼

llama.cpp provides hardware-aware configuration that automatically detects and configures CPU, Metal, CUDA, HIP, and Vulkan backends to achieve optimal local inference performance.

How do I inspect a GGUF file before running local inference?▼

You can inspect a GGUF file to verify model compatibility by using the skill's discovery-first operating procedure, which checks hardware and backend performance before execution.

Why does local LLM inference fail when hardware and model compatibility are not verified?▼

Local LLM inference fails or performs poorly without hardware verification, making a discovery-first procedure essential to configure backends and tune quantization before execution.

Related Skills

Explore More Software Engineering Skills→
davpatel605-beepdavpatel605-beep

llama-cpp

Configure llama.cpp deployments and select GGUF quantizations for local inference.

Community
Advanced
hsliuustc0106hsliuustc0106
84

vllm-omni-api

Bridge client applications with the vLLM-Omni API using OpenAI-compatible endpoints.

Community
Intermediate
kwcantrellkwcantrell

llama-server

Serve local GGUF LLMs with an OpenAI-compatible HTTP API.

Community
Advanced
JKhyroJKhyro

llama-cpp

Run local LLM inference with GGUF models on non-NVIDIA hardware.

Community
Advanced