skills.rest
lxt070409
lxt070409COMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local inference with llama.cpp models and discover GGUF models from the Hugging Face Hub.

1|Updated Jun 9, 2026
Search Tags:#GGUF#inference#model discovery#Hugging Face Hub#llama.cpp
One-click install
npx skills add https://github.com/lxt070409/hermes-skills --skill llama-cpp-lxt070409
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/lxt070409/hermes-skills/tree/main/mlops/inference/llama-cpp
Command: npx skills add https://github.com/lxt070409/hermes-skills --skill llama-cpp-lxt070409

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill solves the problem of local inference using llama.cpp models and the discovery of models from the Hugging Face Hub, enabling users to run models on various hardware and discover models that support llama.cpp.

Core Features & Use Cases

  • Local GGUF Inference: Perform inference using llama.cpp models directly on the local machine.
  • Model Discovery: Discover and select GGUF models from the Hugging Face Hub for local inference.
  • Use Case: A user wants to use a specific model for inference that is not installed locally. This Skill allows the user to find the model, download the GGUF file, and use it for inference without installing the model directly.

Quick Start

Use the llama-cpp skill to run the GGUF file from the Hugging Face Hub for the 'bartowski/Llama-3.2-3B-Instruct-GGUF' model with the Q8_0 quantization.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using a model from the Hugging Face Hub?▼

You can run local GGUF inference by discovering a model on the Hugging Face Hub, downloading its GGUF file, and executing it directly via the llama.cpp framework without needing a full model installation.

Can I use llama.cpp for inference on Apple Silicon and CUDA GPUs?▼

Yes, llama.cpp inference supports multiple hardware backends including Apple Silicon, CUDA, ROCm, and Intel GPUs. This allows local GGUF model execution across diverse hardware environments.

What is the best way to find Hugging Face Hub models that support llama.cpp?▼

The best way to find compatible models is using the model discovery feature, which specifically searches the Hugging Face Hub for GGUF files that support llama.cpp execution for your local environment.

Do I need to install a model locally before running inference with llama.cpp?▼

No, you do not need to pre-install the model locally. You can discover a specific model on the Hugging Face Hub, download the GGUF file, and use it for inference directly.

What are the dependencies required to perform local GGUF inference?▼

Performing local GGUF inference requires the llama-cpp-python dependency, along with various hardware-specific components and libraries to support CPU, CUDA, ROCm, or Intel GPU execution.

Why use GGUF files for local inference instead of standard model formats?▼

GGUF files are optimized for llama.cpp, allowing efficient local inference across different hardware like CPU and Apple Silicon. They enable direct execution from Hugging Face Hub downloads without complex installation.

Related Skills

Explore More Software Engineering Skills→
AlexiosBluffMaraAlexiosBluffMara
2

llama-cpp

Run local GGUF inference and discover models via Hugging Face Hub.

Community
Intermediate
EurecaMomentEurecaMoment

llm-local-qwen

Deploy qwen3.5-0.8b locally via vLLM for text generation and chat completion.

Official
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
mahg-esmahg-es
3

llm-local-deploy

Deploy local LLMs with llama.cpp, Ollama, or vLLM via docker-compose.

Official
Intermediate