skills.rest
AissenLiu
AissenLiuCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run GGUF-based llama.cpp inference locally and discover compatible Hugging Face models.

Updated Apr 27, 2026
Search Tags:#inference#huggingface#llama-cpp#model-discovery#gguf#local-deployment
One-click install
npx skills add https://github.com/AissenLiu/EasyHermes --skill llama-cpp-aissenliu
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/AissenLiu/EasyHermes/tree/main/hermes-agent/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/AissenLiu/EasyHermes --skill llama-cpp-aissenliu

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0.

What problem does it solve?

Locally run GGUF-based inference for llama.cpp and discover compatible Hugging Face models without relying on external API services.

Core Features & Use Cases

  • Local GGUF inference on CPU or GPU with seamless HF Hub model discovery.
  • Quant selection guidance and quick deployment workflows for testing models offline.
  • Simple server/client setup enabling rapid iteration between models and prompts.

Quick Start

Launch the llama.cpp server with a suitable GGUF file from Hugging Face to run local inference.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference offline using llama.cpp?▼

Find compatible GGUF files via Hugging Face Hub and launch them locally using llama-server or llama-cli. This enables offline testing of models on CPU or GPU hardware.

Can I use llama-cpp-python to discover Hugging Face models for local deployment?▼

Yes, it integrates Hugging Face Hub model discovery to find compatible GGUF files for local deployment. This streamlines selecting quantized models suitable for CPU or GPU execution.

Does local GGUF inference work on both CPU and GPU?▼

Local GGUF inference supports both CPU and GPU deployments. You can select appropriate quantization levels and configure llama-server to test models on available hardware.

What is the best way to set up a server-client workflow for offline model testing?▼

Launch the llama-server with a suitable GGUF file and connect a local client. This setup enables rapid iteration between models and prompts for offline testing.

How do I choose the right quantization for a GGUF model?▼

Selecting GGUF quantization depends on hardware memory constraints. The skill provides quant selection guidance to balance inference speed and model accuracy for local deployment.

Why use llama.cpp for local inference instead of external API services?▼

Using llama.cpp for local inference removes dependency on external API services and enables offline model testing. It provides full control over GGUF quant selection and data privacy.

Related Skills

Explore More Software Engineering Skills→
eliottbusinesseliottbusiness

llama-cpp

Run llama.cpp GGUF model inference locally with Hugging Face Hub discovery.

Community
Advanced
ovachieverovachiever
52

llama-cpp

Run LLM inference on CPU and non-NVIDIA hardware with GGUF quantization.

Community
Advanced
SignmanalSignmanal
1

llama-cpp

Run local GGUF model inference with llama.cpp across CPU and GPU hardware.

Community
Intermediate
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced