skills.rest
Brilly-Bohyun
Brilly-BohyunCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Run Hugging Face Hub GGUF models locally with llama.cpp across CPU and GPU platforms.

Updated Jun 17, 2026
Search Tags:#GGUF#quantization#inference#model discovery#Hugging Face Hub
One-click install
npx skills add https://github.com/Brilly-Bohyun/skill-repository --skill llama-cpp-brilly-bohyun
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/Brilly-Bohyun/skill-repository/tree/main/mlops/inference/llama-cpp
Command: npx skills add https://github.com/Brilly-Bohyun/skill-repository --skill llama-cpp-brilly-bohyun

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill simplifies local GGUF inference using llama.cpp, enabling users to run Hugging Face Hub models with high efficiency and flexibility on various platforms.

Core Features & Use Cases

  • Local GGUF Inference: Perform inference on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.
  • Model Discovery: Discover and select the appropriate Hugging Face Hub model for llama.cpp.
  • Quantization: Choose and apply different quantization levels for optimal performance and memory usage.
  • Use Case: Imagine you have a specific research task requiring a highly optimized model. Use this Skill to quickly identify and download the right model with the desired quantization level from the Hugging Face Hub.

Quick Start

Use the llama-cpp skill to run the Hugging Face Hub model 'bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0' locally on your machine.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run Hugging Face Hub models locally using GGUF inference?▼

To run Hugging Face Hub models locally, you can use this Skill to facilitate GGUF inference via llama.cpp, allowing you to download and execute models directly on your machine.

Can I use llama.cpp for local inference on Apple Silicon or CUDA GPUs?▼

Yes, local inference is supported across multiple platforms including CPU, Apple Silicon, CUDA, ROCm, and Intel GPUs to ensure flexible hardware compatibility.

What is the best way to optimize memory usage during local GGUF inference?▼

The best way to optimize memory is by applying different quantization levels to your models, which allows for optimal performance and reduced memory footprint.

How do I discover the right Hugging Face Hub model for llama.cpp?▼

You can discover and select the appropriate Hugging Face Hub model for your specific research task using the model discovery features integrated into the Skill.

Does this Skill require a specific version of llama-cpp-python?▼

Yes, this Skill requires llama-cpp-python version 0.2.0 or higher to perform GGUF inference and interact with Hugging Face Hub models.

What is a quick start example for running a GGUF model locally?▼

A quick start example is running the Hugging Face Hub model 'bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0' locally on your machine to test GGUF inference.

Related Skills

Explore More Software Engineering Skills→
agentic-inagentic-in
577

llama.cpp

Run local LLM inference with llama.cpp across CPU, Apple Silicon, and GPUs.

Official
Advanced
JuanMS20JuanMS20

serving-llms-vllm

Deploy high-throughput LLM serving endpoints with vLLM and OpenAI-compatible APIs.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
juliuss1907juliuss1907

llama-cpp

Run local GGUF inference with llama.cpp models from Hugging Face Hub.

Community
Advanced