skills.rest
ashiqcodeleaf
ashiqcodeleafCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run local GGUF inference with llama.cpp and discover Hugging Face Hub models.

Updated Jun 22, 2026
Search Tags:#GGUF#inference#Hugging Face Hub#llama.cpp#local model
One-click install
npx skills add https://github.com/ashiqcodeleaf/long-Run-Agents --skill llama-cpp-ashiqcodeleaf
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/ashiqcodeleaf/long-Run-Agents/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/ashiqcodeleaf/long-Run-Agents --skill llama-cpp-ashiqcodeleaf

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires ggml, llama.cpp, python, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides local GGUF inference and Hugging Face Hub model discovery for llama.cpp, allowing users to leverage the power of llama.cpp for various inference tasks.

Core Features & Use Cases

  • Local GGUF Inference: Run llama.cpp models locally on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.
  • Hugging Face Hub Model Discovery: Search for and download GGUF files for llama.cpp models directly from the Hugging Face Hub.
  • Use Case: When you need to perform fast inference on a llama.cpp model without the need for an internet connection or when you require specific model parameters not available on the Hub.

Quick Start

Use the llama-cpp skill to run the llama.cpp model 'bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0' locally for inference.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference offline using llama.cpp?▼

Run local GGUF inference by executing llama.cpp models directly on your hardware. This allows fast offline inference on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs without requiring an internet connection.

How do I discover and download GGUF models from the Hugging Face Hub?▼

Discover GGUF models from the Hugging Face Hub using URL-based workflows. You can search for specific model parameters and download GGUF files directly to run locally with llama.cpp.

Do I need Python and ggml installed to run llama.cpp models locally?▼

Yes, you need Python, ggml, and llama.cpp installed to run models locally. These dependencies are required to execute the inference scripts and manage the model parameters on your system.

Can I use llama.cpp for inference on Apple Silicon and CUDA GPUs?▼

Yes, llama.cpp supports local inference across multiple hardware backends. You can run GGUF models on Apple Silicon, CUDA, ROCm, Intel GPUs, or standard CPUs to leverage your available computing resources.

What's the best way to perform inference on a specific Llama-3.2 GGUF model?▼

To perform inference on a specific model like Llama-3.2-3B-Instruct-GGUF, download the desired parameter file from the Hugging Face Hub and execute it locally using the llama.cpp scripts.

When should I choose local GGUF inference over standard Hugging Face Hub model execution?▼

Choose local GGUF inference when you need offline execution capabilities or require specific model parameters not readily available through standard Hugging Face Hub model execution workflows.

Related Skills

Explore More Software Engineering Skills→
richardnguyen0715richardnguyen0715

llama-cpp

Execute local GGUF inference for llama.cpp models across multiple hardware platforms.

Community
Advanced
mupengi-botmupengi-bot
11

data-scraper

Fetch HTML from URLs and extract structured data into JSON or Markdown.

Community
Intermediate
aivos-xieaivos-xie
1

llama-cpp

Run local GGUF inference with llama.cpp models from the Hugging Face Hub.

Community
Intermediate
hochoa13hochoa13

llama-cpp

Run GGUF quantized LLM inference on CPUs, Apple Silicon, and consumer GPUs.

Community
Intermediate