skills.rest
519lab
519labOFFICIAL
Explore all skills by this author →
WebsiteGitHub

llama-cpp

Discover and run llama.cpp GGUF models locally via Hugging Face Hub.

2|Updated May 22, 2026
Search Tags:#model discovery#CPU inference#Hugging Face Hub#llama.cpp#GGUF inference
One-click install
npx skills add https://github.com/519lab/thoth-agent --skill llama-cpp-519lab
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/519lab/thoth-agent/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/519lab/thoth-agent --skill llama-cpp-519lab

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenges of finding, discovering, and running llama.cpp models for various inference tasks on different platforms, simplifying the process of local GGUF inference.

Core Features & Use Cases

  • Model Discovery: Find suitable llama.cpp models from the Hugging Face Hub based on specific criteria.
  • Local Inference: Perform local GGUF inference with models downloaded directly to your machine.
  • Use Case: If you are working with llama.cpp models for CPU or GPU inference, this skill helps you discover and set up models without leaving your terminal.

Quick Start

Install the skill using the command pip install llama-cpp-discovery and then run llama-cpp-discovery search <model_name> to find a model.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference with llama.cpp models?▼

Local GGUF inference is performed by using this skill to run downloaded GGUF files directly on your machine. It requires the llama-cpp-python library and GGUF-compatible models to execute inference tasks.

How do I discover llama.cpp models on the Hugging Face Hub?▼

Model discovery on the Hugging Face Hub is achieved by searching for llama.cpp models based on specific criteria. The skill enables users to find suitable models and view hardware compatibility without leaving the terminal.

Does llama-cpp-python support CPU inference for GGUF files?▼

Yes, llama-cpp-python supports CPU inference for GGUF files. The skill provides a comprehensive workflow for discovering and running llama.cpp models on local machines for various CPU or GPU inference tasks.

What is the best way to find hardware-compatible llama.cpp models?▼

To find hardware-compatible llama.cpp models, use the model discovery feature to search the Hugging Face Hub. The workflow allows you to search for models and view hardware compatibility directly from your terminal.

Do I need specific dependencies to run local GGUF inference?▼

Yes, you need the llama-cpp-python dependency, specifically version 0.2.0 or higher, to run local GGUF inference. You also need to download GGUF-compatible models to your local machine.

Can I search for Hugging Face Hub models from my terminal?▼

Yes, you can search the Hugging Face Hub for llama.cpp models directly from your terminal. After installing the skill, run the search command with a model name to discover suitable models for local inference.

Related Skills

Explore More Software Engineering Skills→
AlexiosBluffMaraAlexiosBluffMara
2

llama-cpp

Run local GGUF inference and discover models via Hugging Face Hub.

Community
Intermediate
ScientiaCapitalScientiaCapital
1

model-deployment

Export fine-tuned models to GGUF and deploy with vLLM or Ollama.

Community
Advanced
aivos-xieaivos-xie
1

llama-cpp

Run local GGUF inference with llama.cpp models from the Hugging Face Hub.

Community
Intermediate
dennisonbertramdennisonbertram
1

ollama-ops

Orchestrate local LLM inference and model management with Ollama.

Community
Intermediate