skills.rest
bailynlove
bailynloveCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

llama-cpp

Run llama.cpp models locally and discover GGUF models from Hugging Face Hub.

1|Updated Apr 29, 2026
Search Tags:#inference#model discovery#Hugging Face Hub#llama.cpp#local execution
One-click install
npx skills add https://github.com/bailynlove/STARK-TOWER --skill llama-cpp-bailynlove
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/bailynlove/STARK-TOWER/tree/main/opencrew/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/bailynlove/STARK-TOWER --skill llama-cpp-bailynlove

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill enables local inference with llama.cpp models and discovery of models from the Hugging Face Hub, eliminating the need for cloud access and reducing latency.

Core Features & Use Cases

  • Local Inference: Run llama.cpp models on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs for lower latency and privacy.
  • Model Discovery: Find and download GGUF models directly from the Hugging Face Hub without the need for additional tools.
  • Use Case: Imagine you want to run a language model on a local device without internet access. This Skill allows you to find a suitable model from the Hub and execute it locally.

Quick Start

Use the llama-cpp skill to run the model 'bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0' locally.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run GGUF models locally for inference without cloud access?▼

To run GGUF models locally, this Skill enables local inference with llama.cpp, allowing you to execute models on your hardware and discover new models directly from the Hugging Face Hub without internet access.

Can I run llama.cpp inference on Apple Silicon or CUDA GPUs?▼

Yes, local llama.cpp inference supports various hardware configurations including Apple Silicon, CUDA, ROCm, Intel GPUs, and CPU, allowing you to run models with lower latency and enhanced privacy.

How do I discover and download GGUF models from the Hugging Face Hub?▼

You can discover and download GGUF models directly from the Hugging Face Hub using this Skill, which eliminates the need for additional tools by integrating model discovery into your local execution workflow.

Do I need Python and llama.cpp to execute models locally?▼

Yes, you need llama.cpp to execute models and Python for scripting, as this Skill relies on the llama-cpp-python dependency to provide local inference capabilities.

What is the best way to run a specific Hugging Face model like Llama-3.2-3B-Instruct-GGUF locally?▼

To run a specific model like Llama-3.2-3B-Instruct-GGUF locally, use this Skill to download the GGUF format from the Hub and execute it directly on your local system hardware.

Related Skills

Explore More Software Engineering Skills→
ToqsickToqsick

llama-cpp

Run local GGUF inference for llama.cpp models with quantization selection and Hugging Face Hub discovery.

Community
Advanced
arendon1arendon1

document-manager

Automates document reading, conversion, extraction, OCR, and merging across PDF, DOCX, PPTX, XLSX, and Markdown.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
winsorllcwinsorllc

local-llm-provider

Connect to local LLM endpoints with automatic cloud fallback.

Community
Advanced