skills.rest
DarkArty07
DarkArty07COMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Deploy local LLM inference with GGUF quantized weights via llama.cpp.

3|Updated Apr 21, 2026
Search Tags:#quantization#llama-cpp#model-discovery#gguf#hugging-face#local-inference#llm-deployment
One-click install
npx skills add https://github.com/DarkArty07/Aether-Agents --skill llama-cpp-darkarty07
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/DarkArty07/Aether-Agents/tree/main/home/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/DarkArty07/Aether-Agents --skill llama-cpp-darkarty07

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This skill eliminates the friction of setting up local LLM inference, selecting the right quantized GGUF model for your hardware, and finding compatible models on the Hugging Face Hub without manual trial and error.

Core Features & Use Cases

  • Local GGUF Inference: Run LLMs on CPU, Apple Silicon, NVIDIA, AMD, or Intel GPUs using llama.cpp or its Python bindings.
  • HF Hub Model Discovery: Find pre-quantized GGUF models, retrieve exact launch commands, and match quantization levels to your available RAM or VRAM.
  • Use Case: If you have a 16GB RAM laptop and want to run a 7B parameter chat model, this skill finds the optimal Q4_K_M quant, provides the exact llama-cli command, and confirms the model file exists on the Hub.

Quick Start

Use the llama-cpp skill to locate a compatible GGUF model for your hardware and get the exact command to run it locally.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local LLM inference using GGUF files on my specific hardware?▼

Run local LLM inference by executing quantized GGUF weights via llama.cpp commands, which supports CPU, Apple Silicon, NVIDIA, AMD, and Intel GPU hardware configurations.

What's the best way to find a compatible GGUF model on Hugging Face for my available RAM?▼

Find compatible GGUF models on Hugging Face by matching quantization levels to your available RAM or VRAM, ensuring the selected model file exists and retrieving the exact launch command.

How do I select the right quantization level for a 7B parameter model on a 16GB laptop?▼

Select the right quantization level for a 7B model on a 16GB laptop by evaluating hardware constraints to find an optimal fit like Q4_K_M, balancing memory limits and performance.

Does llama.cpp support local deployment on both CPU and Apple Silicon GPUs?▼

Yes, llama.cpp supports local deployment across CPU, Apple Silicon, NVIDIA, AMD, and Intel GPUs, allowing flexible inference execution based on your available hardware environment.

What are the limitations when running quantized models locally with GGUF?▼

Limitations when running quantized GGUF models locally include strict hardware memory constraints requiring careful quantization matching, alongside potential performance optimization and troubleshooting workflow challenges.

Can I convert models to GGUF format and optimize performance for local server-based inference?▼

Yes, you can convert models to GGUF format and construct server-based inference commands, utilizing provided performance optimization and troubleshooting workflows to ensure efficient local deployment.

Related Skills

Explore More Software Engineering Skills→
Tyler-R-KendrickTyler-R-Kendrick
1

huggingface-local-models

Search Hugging Face Hub and launch GGUF models via llama.cpp.

Community
Intermediate
SignmanalSignmanal
1

llama-cpp

Run local GGUF model inference with llama.cpp across CPU and GPU hardware.

Community
Intermediate
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
afel6afel6

llama-cpp

Run GGUF-quantized LLM inference on CPUs and non-NVIDIA GPUs.

Community
Advanced