skills.rest
AVOI-CEO
AVOI-CEOCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local LLM inference with llama.cpp across CPU, Apple Silicon, and GPUs.

Updated Aug 23, 2026
Search Tags:#server#quantization#inference#edge#gguf#llama.cpp#python-bindings
One-click install
npx skills add https://github.com/AVOI-CEO/avoi-agent --skill llama-cpp-avoi-ceo
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/AVOI-CEO/avoi-agent/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/AVOI-CEO/avoi-agent --skill llama-cpp-avoi-ceo

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Enables local LLM inference across CPU, Apple Silicon, and GPUs using llama.cpp, including GGUF conversion and quantization to support edge deployments and offline workflows.

Core Features & Use Cases

  • CLI-driven and Python bindings for flexible integration with existing pipelines.
  • OpenAI-compatible server support and Ollama/LM Studio integrations for seamless deployment.
  • Supports quantization (2–8 bit) and model conversion to GGUF for performance on diverse hardware.

Quick Start

Install llama.cpp, select a build backend, and start local inference with optional GGUF quantization and a server.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local LLM inference on CPU and Apple Silicon?▼

Local LLM inference on CPU and Apple Silicon uses llama.cpp with GGML or Metal backends, optionally leveraging llama-cpp-python for programmatic prompts to execute offline workflows.

What is GGUF conversion and quantization for local models?▼

GGUF conversion and quantization reduce model bit-width to 2-8 bit, enabling efficient local LLM inference on edge deployments. This process optimizes model size and performance for diverse hardware targets.

Does llama.cpp support OpenAI-compatible server setups?▼

llama.cpp supports OpenAI-compatible server setups for local LLM inference, allowing seamless deployment and integration with Ollama and LM Studio workflows.

Can I use Python bindings for local LLM inference?▼

Use the llama-cpp-python Python bindings to run programmatic prompts for local LLM inference, enabling flexible CLI-driven integration with existing CPU and GPU pipelines.

What do I need to set up an edge deployment for local LLMs?▼

Edge deployments for local LLMs require a compatible build environment with CPU or GPU backends like GGML, Metal, or CUDA, plus GGUF conversion and 2-8 bit quantization for offline performance.

How do I convert models to GGUF format for offline workflows?▼

Convert models to GGUF format using llama.cpp utilities with optional 2-8 bit quantization to optimize performance for offline workflows and edge deployments across varied hardware backends.

Related Skills

Explore More Software Engineering Skills→
KarlinskySKarlinskyS

llama-cpp

Run transformer language models on CPU and non-NVIDIA GPUs with GGUF quantization.

Community
Intermediate
lucasfthlucasfth

llama-cpp

Run local LLM inference and quantize GGUF models with llama.cpp.

Community
Advanced
AlexiosBluffMaraAlexiosBluffMara
2

llama-cpp

Run local GGUF inference and discover models via Hugging Face Hub.

Community
Intermediate
hochoa13hochoa13

llama-cpp

Run GGUF quantized LLM inference on CPUs, Apple Silicon, and consumer GPUs.

Community
Intermediate