skills.rest
servitola
servitolaCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Guide GGUF quant selection and launch llama.cpp server or CLI inference.

12|2|Updated Feb 2, 2020
Search Tags:#server#quantization#huggingface#gguf#llama.cpp#local-inference#llama-server
One-click install
npx skills add https://github.com/servitola/dotfiles --skill llama-cpp-servitola
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/servitola/dotfiles/tree/main/claude-code/skills/llama-cpp
Command: npx skills add https://github.com/servitola/dotfiles --skill llama-cpp-servitola

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Simplifies running local GGUF models by providing a guided workflow to pick the right quant, locate compatible GGUF files on Hugging Face, and launch a local inference server or CLI with minimal setup.

Core Features & Use Cases

  • Local inference with llama.cpp on CPU, Apple Silicon, CUDA, and ROCm systems.
  • Quant discovery and selection using Hugging Face repository metadata.
  • Quick starts for server or CLI-based deployment and model evaluation.

Quick Start

Install llama.cpp, select a suitable quant, and start a local server or run llama-cli to begin local inference.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF models using llama.cpp for inference?▼

Run local GGUF models by installing llama.cpp, selecting a compatible quant from Hugging Face metadata, and launching a local inference server or CLI to start generating text on your machine.

What is the best way to choose a quant for local inference on Apple Silicon or CUDA?▼

Choosing a quant for local inference involves evaluating Hugging Face repository metadata to find the right balance of model size and precision for your CPU, Apple Silicon, CUDA, or ROCm hardware environment.

Does llama.cpp work across CPU, Apple Silicon, CUDA, and ROCm environments?▼

Yes, llama.cpp works across CPU, Apple Silicon, CUDA, and ROCm environments, providing a guided workflow to deploy local GGUF model inference and launch a local server regardless of your specific hardware setup.

How do I find and download GGUF files from Hugging Face for llama.cpp?▼

You find GGUF files for llama.cpp by using quant discovery techniques that parse Hugging Face repository metadata to locate compatible model files for your local deployment and evaluation needs.

Can I launch a local llama-server instead of using the CLI for model evaluation?▼

Yes, you can launch a local llama-server instead of the CLI. This Skill provides quick starts for both server and CLI-based deployment to evaluate local GGUF models with minimal setup overhead.

Related Skills

Explore More Software Engineering Skills→
fikriaffikriaf

llama-cpp

Automate discovery, download, and local inference of llama.cpp models from Hugging Face Hub.

Community
Advanced
zengbaochengzengbaocheng

llama-cpp

Run local GGUF inference with llama.cpp using llama-server or llama-cli.

Community
Advanced
eliottbusinesseliottbusiness

llama-cpp

Run llama.cpp GGUF model inference locally with Hugging Face Hub discovery.

Community
Advanced
SignmanalSignmanal
1

llama-cpp

Run local GGUF model inference with llama.cpp across CPU and GPU hardware.

Community
Intermediate