skills.rest
zaiyemeiyou404
zaiyemeiyou404COMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local GGUF inference with llama.cpp and discover models from Hugging Face Hub.

Updated May 25, 2026
Search Tags:#model discovery#CPU inference#Hugging Face Hub#llama.cpp#GGUF inference
One-click install
npx skills add https://github.com/zaiyemeiyou404/Hermes --skill llama-cpp-zaiyemeiyou404
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/zaiyemeiyou404/Hermes/tree/main/backup/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/zaiyemeiyou404/Hermes --skill llama-cpp-zaiyemeiyou404

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill solves the problem of local GGUF inference using llama.cpp, and allows for discovery of Hugging Face Hub models that support llama.cpp, optimizing inference on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.

Core Features & Use Cases

  • Local GGUF Inference: Execute llama.cpp models locally for inference.
  • Hugging Face Hub Model Discovery: Search and select appropriate models from Hugging Face Hub that support llama.cpp.
  • Use Case: A developer looking to deploy a llama.cpp based application that requires efficient CPU inference with the ability to dynamically load models from the Hugging Face Hub.

Quick Start

Install llama.cpp and use the llama-cli or llama-server commands to infer from a local GGUF file. For example, to run a model from the Hub, you can use:

llama-cli -hf <repo>:<QUANT>

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp?▼

Local GGUF inference using llama.cpp is executed via the llama-cli or llama-server commands. You can run a model directly from the Hugging Face Hub by specifying the repository and quantization parameters using the -hf flag.

Does llama.cpp support CPU inference for models discovered on the Hugging Face Hub?▼

Yes, llama.cpp supports CPU inference for models discovered on the Hugging Face Hub. It enables efficient model retrieval and quantization selection to optimize inference across various hardware platforms.

Can I use llama.cpp for inference on Apple Silicon or CUDA GPUs?▼

Yes, you can use llama.cpp for inference on Apple Silicon and CUDA GPUs. The Skill facilitates quantization selection and model retrieval to optimize local inference across CPU, Apple Silicon, CUDA, ROCm, and Intel GPUs.

What is the best way to discover Hugging Face Hub models that support llama.cpp?▼

The best way to discover compatible Hugging Face Hub models is using this Skill's model discovery feature. It allows you to search and select appropriate models from the Hub that support local GGUF inference.

Do I need llama-cpp-python bindings to run local GGUF models?▼

Yes, you need the llama-cpp-python bindings installed at version 0.2.0 or higher. This Python binding is required to access Hugging Face Hub models and execute local GGUF inference.

How do I select the right quantization when loading a model from the Hugging Face Hub?▼

You select the desired quantization by appending the quantization name to the repository identifier. When using the llama-cli command, specify the repository and quantization format as -hf <repo>:<QUANT>.

Related Skills

Explore More Software Engineering Skills→
sheawinklersheawinkler
78

llama-cpp

Run local GGUF inference and discover Hugging Face Hub models with llama.cpp.

Community
Advanced
ovachieverovachiever
52

llama-cpp

Run LLM inference on CPU and non-NVIDIA hardware with GGUF quantization.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
eyadsibaieyadsibai
7

llm-inference

Compare inference engines like vLLM and llama.cpp for LLM serving.

Community
Intermediate