skills.rest
AlexKoncept
AlexKonceptCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

llama-cpp

Locates GGUF files on Hugging Face Hub and launches a local llama.cpp inference server for testing.

Updated Aug 27, 2026
Search Tags:#inference#huggingface#llama-cpp#model-discovery#gguf#local-execution
One-click install
npx skills add https://github.com/AlexKoncept/omnia-hub --skill llama-cpp-alexkoncept
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/AlexKoncept/omnia-hub/tree/main/HERMES/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/AlexKoncept/omnia-hub --skill llama-cpp-alexkoncept

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Enables fast, private local inference for llama.cpp and seamless discovery of GGUF models on the Hugging Face Hub.

Core Features & Use Cases

  • Local GGUF inference on CPU or GPU with support for quantization and hardware offload.
  • Model discovery and selection from Hugging Face Hub using the local-app and tree API workflows to identify exact filenames and sizes.
  • Deployment guidance for running an OpenAI-compatible server or client using llama-server or llama-cli.

Quick Start

Install the llama.cpp toolchain, choose a GGUF model from the Hub, and start local inference with the server or CLI.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp?▼

Run local GGUF inference by installing the llama.cpp toolchain, selecting a model from Hugging Face Hub, and launching the OpenAI-compatible server or CLI for immediate testing on your CPU or GPU.

What is the best way to find compatible GGUF models on Hugging Face?▼

Find compatible GGUF models on Hugging Face by using model discovery workflows that query the Hub API to identify exact filenames and sizes for your local execution environment.

Can I use llama.cpp for local inference on both CPU and GPU?▼

Yes, local inference works on both CPU and GPU environments with support for hardware offload and quantization options like Q4_K_M to optimize performance during execution.

Do I need an internet connection to use local GGUF inference?▼

You need internet access to download GGUF models from Hugging Face Hub, but once downloaded, local GGUF inference runs entirely offline, ensuring privacy-preserving execution without external API calls.

How do I evaluate quantization options like Q4_K_M for local execution?▼

Evaluate quantization options like Q4_K_M by checking the available GGUF file sizes during model discovery on Hugging Face Hub to balance inference speed and memory usage for your hardware.

Does llama.cpp provide an OpenAI-compatible server for local testing?▼

Yes, llama.cpp provides an OpenAI-compatible server using llama-server, allowing you to deploy local models for immediate testing and integration with existing API clients.

Related Skills

Explore More Software Engineering Skills→
samuelmukotisamuelmukoti

llama-cpp

Run local LLM inference with llama.cpp and GGUF models on CPU or GPU.

Community
Advanced
jayll1303jayll1303
17

llama-cpp-inference

Run local GGUF model inference and OpenAI-compatible serving with llama.cpp.

Community
Advanced
SriRamkunamsettySriRamkunamsetty

llama-cpp

Run local GGUF inference and discover Hugging Face models for llama.cpp.

Community
Intermediate
ovachieverovachiever
52

llama-cpp

Run LLM inference on CPU and non-NVIDIA hardware with GGUF quantization.

Community
Advanced