skills.rest
heximao
heximaoCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Find and deploy llama.cpp GGUF models from the Hugging Face Hub for local inference.

Updated May 11, 2026
Search Tags:#AI deployment#GGUF#Hugging Face Hub#llama.cpp#model inference
One-click install
npx skills add https://github.com/heximao/agents --skill llama-cpp-heximao
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/heximao/agents/tree/main/skill-archive/docker-hermes/mlops/inference/llama-cpp
Command: npx skills add https://github.com/heximao/agents --skill llama-cpp-heximao

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines local GGUF inference with llama.cpp and simplifies the discovery of models from the Hugging Face Hub, reducing the complexity of deploying and managing AI models.

Core Features & Use Cases

  • Local GGUF Inference: Execute llama.cpp models on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.
  • Model Discovery: Explore and select the right GGUF model for a specific Hugging Face repository.
  • Quick Server Setup: Build a llama-server or llama-cli command from the Hub with minimal configuration.
  • Use Case: A user looking for a model to perform Q&A sessions with a specific size and hardware compatibility can use this Skill to quickly identify the appropriate model and configure it for use.

Quick Start

Use the llama-cpp skill to find and run the llama.cpp model with the desired quantization level from the Hugging Face Hub. For example: 'use llama-cpp to run bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0'.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF model inference using llama.cpp?▼

Run local GGUF inference by configuring llama.cpp Python bindings to execute models directly on your CPU, Apple Silicon, CUDA, ROCm, or Intel GPU. This Skill streamlines the deployment process for efficient hardware-based model execution.

How do I find and download the right GGUF models from the Hugging Face Hub?▼

Find and download GGUF models from the Hugging Face Hub by using this Skill to explore repositories and select specific quantization levels. It simplifies discovering models that match your exact size and hardware compatibility requirements.

What's the best way to set up a llama-server or llama-cli from the Hugging Face Hub?▼

Set up a llama-server or llama-cli by using this Skill to generate the required command from a Hugging Face Hub repository with minimal configuration. It allows you to quickly launch inference servers using your selected GGUF model.

Does this approach support running GGUF models across different hardware like Apple Silicon and CUDA?▼

Yes, this approach supports running GGUF models across diverse hardware. You can execute llama.cpp models efficiently using CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs, ensuring flexible local deployment based on your available hardware.

Do I need specific Python bindings to deploy llama.cpp models locally?▼

Yes, you need the llama-cpp-python bindings installed to deploy llama.cpp models locally. This Skill requires these bindings along with Hugging Face Hub access to streamline model discovery and hardware-based execution.

Related Skills

Explore More Software Engineering Skills→
sheawinklersheawinkler
78

llama-cpp

Run local GGUF inference and discover Hugging Face Hub models with llama.cpp.

Community
Advanced
BermudaLocalsBermudaLocals
1

llama-cpp

Execute local GGUF inference and discover Hugging Face models with llama.cpp.

Community
Intermediate
DonZzzillaDonZzzilla

llama-cpp

Run local GGUF inference for llama.cpp models with Hugging Face Hub discovery.

Community
Advanced
hochoa13hochoa13

llama-cpp

Run GGUF quantized LLM inference on CPUs, Apple Silicon, and consumer GPUs.

Community
Intermediate