skills.rest
jordanhubbard
jordanhubbardCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

llama-cpp

Run inference on local GGUF models using llama.cpp Python bindings.

9|3|Updated May 18, 2026
Search Tags:#code generation#AI#GGUF#inference#llama.cpp
One-click install
npx skills add https://github.com/jordanhubbard/mac --skill llama-cpp-jordanhubbard
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/jordanhubbard/mac/tree/main/src/mac/_hermes/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/jordanhubbard/mac --skill llama-cpp-jordanhubbard

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

The llama.cpp skill provides the ability to perform inference on locally available models and repositories using llama.cpp, allowing for efficient analysis and code generation.

Core Features & Use Cases

  • Local GGUF Inference: Utilizes the llama.cpp local model to perform inference on locally available models in the GGUF format.
  • Model Discovery: Navigate the Hugging Face Hub forGGUF model discovery, offering insights into compatibility, quantization, and architecture.
  • Custom GGUF Building: GenerateGGUF models directly from repositories with llama.cpp and its tools.

Quick Start

Use the llama-cpp skill with a command like: llama-cpp --hf-repo bartowski/Llama-3.2-3B-Instruct-GGUF --hf-file Qwen3.6-35B-A3B-UD-Q4_K_M.gguf -c 4096.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local inference with a GGUF model using llama.cpp?▼

Run local GGUF inference by specifying the Hugging Face repository and file in your command, allowing the llama.cpp Python bindings to execute AI-driven analysis and code generation directly on your machine.

Can I search the Hugging Face Hub for GGUF models and build custom ones?▼

Yes, you can navigate the Hugging Face Hub for GGUF model discovery to check compatibility and quantization, and generate custom GGUF models directly from repositories using llama.cpp tools.

Do I need llama-cpp-python to perform local inference on GGUF files?▼

Yes, the llama-cpp-python package is required, as it provides the Python bindings necessary to interact with local GGUF models and perform on-the-fly inference.

What is the best way to use llama.cpp for AI-driven code generation in research workflows?▼

The best way is to use this skill's command-line interface to load a specific GGUF model, enabling optimized on-the-fly AI code generation tailored for research and development workflows.

Why use GGUF format for local model inference instead of other formats?▼

GGUF format enables efficient local inference and custom model building directly from repositories, optimizing compatibility and quantization for research and development workflows.

Related Skills

Explore More Software Engineering Skills→
superfhpsuperfhp

llama-cpp

Run local GGUF inference for llama.cpp models on CPU, Apple Silicon, CUDA, ROCm, and Intel GPUs.

Community
Advanced
jewbetchajewbetcha
115

ux-researcher-designer

Generate data-driven personas, journey maps, and usability testing frameworks from user research data.

Community
Basic
eliottbusinesseliottbusiness

llama-cpp

Run llama.cpp GGUF model inference locally with Hugging Face Hub discovery.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced