skills.rest
Rheasilvia
RheasilviaCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Execute local GGUF model inference and discover Hugging Face models for llama.cpp.

Updated Jun 25, 2026
Search Tags:#llm#quantization#inference#gguf#llama.cpp#hugging face
One-click install
npx skills add https://github.com/Rheasilvia/hermes-desktop --skill llama-cpp-rheasilvia
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/Rheasilvia/hermes-desktop/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/Rheasilvia/hermes-desktop --skill llama-cpp-rheasilvia

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python, and includes references (resource) components.

What problem does it solve?

This skill removes the complexity of running local large language models by providing a unified interface for GGUF inference, quantization selection, and Hugging Face model discovery.

Core Features & Use Cases

  • Model Discovery: Search and filter Hugging Face repositories for llama.cpp compatible models based on hardware constraints and parameter counts.
  • Inference Orchestration: Launch local servers or CLI instances with optimized settings for CPU, Apple Silicon, or GPU acceleration.
  • Use Case: A developer needs to run a 7B parameter model on a laptop with limited RAM; this skill identifies the optimal Q4_K_M quantization and provides the exact command to launch the model via llama-server.

Quick Start

Use the llama-cpp skill to search for trending models and launch a local inference server for the bartowski/Llama-3.2-3B-Instruct-GGUF model using the Q8_0 quantization.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF inference using llama.cpp?▼

Local GGUF inference is executed by reconstructing optimized commands to launch CLI instances or local servers via llama.cpp, supporting diverse hardware configurations including CPU, Apple Silicon, and GPU acceleration.

How do I find Hugging Face models compatible with llama.cpp?▼

Hugging Face model discovery is managed by searching and filtering repositories for llama.cpp compatible models based on specific hardware constraints and parameter counts to ensure optimal local deployment.

Can I run a 7B parameter model on a laptop with limited RAM using llama.cpp?▼

Running a 7B parameter model on limited RAM is supported by identifying the optimal Q4_K_M quantization and providing the exact command to launch the model via llama-server for constrained hardware.

What is the best way to select quantization for GGUF models on Apple Silicon?▼

Quantization selection for GGUF models on Apple Silicon is automated by evaluating hardware constraints and parameter counts to recommend optimized settings for performance tuning and server orchestration.

Does llama-cpp-python support GPU-accelerated deployments for local servers?▼

The llama-cpp-python dependency supports GPU-accelerated deployments alongside CPU and Apple Silicon configurations, enabling optimized local server orchestration for diverse hardware environments.

Why use GGUF format for local LLM inference instead of other formats?▼

GGUF format is used for local LLM inference to enable quantization selection and optimized performance tuning across CPU, Apple Silicon, and GPU environments via the llama.cpp framework.

Related Skills

Explore More Software Engineering Skills→
Ced3-hanCed3-han

llama-cpp

Run LLM inference on CPUs, Apple Silicon, and consumer GPUs with GGUF quantization.

Community
Advanced
x-TheFoxx-TheFox

llama-cpp

Run local GGUF inference across CPU, Apple Silicon, and GPU backends.

Community
Intermediate
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
helix4uhelix4u

llama-cpp

Run LLM inference on CPU, Apple Silicon, and non-NVIDIA GPUs with GGUF models.

Community
Intermediate