skills.rest
superfhp
superfhpCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local GGUF inference for llama.cpp models on CPU, Apple Silicon, CUDA, ROCm, and Intel GPUs.

Updated May 8, 2026
Search Tags:#inference#model discovery#gguf#llama.cpp#hugging face
One-click install
npx skills add https://github.com/superfhp/lumi-agent --skill llama-cpp-superfhp
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/superfhp/lumi-agent/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/superfhp/lumi-agent --skill llama-cpp-superfhp

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill streamlines the process of performing GGUF inference and discovering Hugging Face models compatible with llama.cpp, optimizing for performance and flexibility.

Core Features & Use Cases

  • GGUF Inference: Local GGUF inference on various hardware platforms.
  • Model Discovery: Search and select models from the Hugging Face Hub that support llama.cpp.
  • Quick Start: Convert and run a Hugging Face model from the Hub on the local system without code changes.

Quick Start

Use the llama-cpp skill to run the model 'bartowski/Llama-3.2-3B-Instruct-GGUF' with Q8_0 quantization.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using a Hugging Face model?▼

Local GGUF inference is performed by discovering a compatible Hugging Face model, converting it to the GGUF format if necessary, and executing it directly via llama.cpp bindings without code changes.

Does llama.cpp GGUF inference support Apple Silicon and CUDA GPUs?▼

GGUF inference supports multiple hardware platforms including Apple Silicon, CUDA, ROCm, and Intel GPUs, allowing flexible local execution across different system configurations.

How do I convert Hugging Face models to GGUF for local use?▼

You can convert Hugging Face models to GGUF format for local use by utilizing the custom GGUF conversion capabilities provided, enabling seamless local execution.

Do I need llama-cpp-python to perform GGUF inference?▼

Yes, llama-cpp-python is required as the Python binding dependency to facilitate local GGUF inference and model discovery tasks within the llama.cpp framework.

What is the best way to discover Hugging Face models compatible with llama.cpp?▼

The best way to discover compatible models is using the integrated model discovery feature, which searches the Hugging Face Hub specifically for models supporting llama.cpp inference.

Can I run a quantized Hugging Face model locally without writing code?▼

Yes, you can quickly convert and run a quantized Hugging Face model, such as Q8_0, on your local system using the provided quick start functionality without code changes.

Related Skills

Explore More Software Engineering Skills→
ovachieverovachiever
52

llama-cpp

Run LLM inference on CPU and non-NVIDIA hardware with GGUF quantization.

Community
Advanced
tangzheng202202tangzheng202202
1

llama-cpp

Execute CPU-based LLM inference with GGUF quantization on Apple Silicon and non-NVIDIA GPUs.

Community
Intermediate
DonZzzillaDonZzzilla

llama-cpp

Run local GGUF inference for llama.cpp models with Hugging Face Hub discovery.

Community
Advanced
helix4uhelix4u

llama-cpp

Run LLM inference on CPU, Apple Silicon, and non-NVIDIA GPUs with GGUF models.

Community
Intermediate