skills.rest
Cyapstaye
CyapstayeCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Discover GGUF models on Hugging Face and run local inference with llama.cpp.

9|Updated Jul 1, 2026
Search Tags:#llm#quantization#inference#huggingface#gguf#llama.cpp
One-click install
npx skills add https://github.com/Cyapstaye/Adame_ver.open --skill llama-cpp-cyapstaye
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/Cyapstaye/Adame_ver.open/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/Cyapstaye/Adame_ver.open --skill llama-cpp-cyapstaye

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python, and includes references (resource) components.

What problem does it solve?

This skill simplifies the complex process of discovering, selecting, and running local GGUF models, removing the friction of manual hardware compatibility checks and command-line configuration.

Core Features & Use Cases

  • Model Discovery: Search and filter Hugging Face for models compatible with your hardware constraints.
  • Quantization Guidance: Automatically identify the optimal GGUF quantization level for your specific RAM or VRAM capacity.
  • Server Deployment: Quickly launch an OpenAI-compatible API server for local inference or batch processing.

Quick Start

Use the llama-cpp skill to find and run a quantized version of a specific model from Hugging Face on your local machine.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local GGUF models for inference?▼

You can run local GGUF models by using this skill to discover Hugging Face repositories and execute batched inference workflows through the llama-cpp-python library.

How do I choose the right GGUF quantization level for my hardware?▼

Choosing the right GGUF quantization level requires checking your specific RAM or VRAM capacity, which this skill automates by providing hardware-aware quantization selection for your local machine.

Can I deploy an OpenAI-compatible API server using llama.cpp?▼

Yes, you can deploy an OpenAI-compatible API server using this skill to facilitate local inference and batch processing workflows directly on your compatible hardware.

Do I need specific drivers to run GGUF models with CPU or GPU acceleration?▼

Yes, running GGUF models with CPU or GPU acceleration requires compatible hardware drivers alongside the llama-cpp-python dependency to function correctly.

What is the best way to find Hugging Face models compatible with my hardware constraints?▼

The best way to find compatible models is using the model discovery feature, which allows you to search and filter Hugging Face repositories based on your specific hardware constraints.

Related Skills

Explore More Software Engineering Skills→
anxiety135790anxiety135790

llama-cpp

Build exact llama.cpp launch commands for compatible GGUF models.

Community
Advanced
kwcantrellkwcantrell

llama-server

Serve local GGUF LLMs with an OpenAI-compatible HTTP API.

Community
Advanced
webdevtodayjasonwebdevtodayjason

llama-cpp

Automate local GGUF inference and model discovery for llama.cpp.

Community
Advanced
J4JoshuaJ4Joshua

mlx

Optimize local AI inference on Apple Silicon using MLX acceleration.

Community
Advanced