skills.rest
fikriaf
fikriafCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

llama-cpp

Automate discovery, download, and local inference of llama.cpp models from Hugging Face Hub.

Updated Apr 29, 2026
Search Tags:#GGUF#model selection#model discovery#llama.cpp#local inference
One-click install
npx skills add https://github.com/fikriaf/agentos --skill llama-cpp-fikriaf
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/fikriaf/agentos/tree/main/src/agentos/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/fikriaf/agentos --skill llama-cpp-fikriaf

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python>=0.2.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill simplifies the process of finding, downloading, and running llama.cpp models, including local inference and model selection.

Core Features & Use Cases

  • Model Discovery: Search and discover llama.cpp models on the Hugging Face Hub.
  • Local Inference: Run llama.cpp models locally on various hardware platforms (CPU, Apple Silicon, CUDA, ROCm, Intel GPUs).
  • Model Selection: Compare and select the right GGUF file based on model size, precision, and compatibility.
  • Use Case: If you're working on a resource-constrained device, this Skill helps you quickly identify a llama.cpp model that fits your memory constraints and still delivers acceptable performance.

Quick Start

Use the llama-cpp skill to run the llama.cpp model 'bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0' locally on your CPU.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local inference with llama.cpp models on my CPU?▼

Run llama.cpp local inference by automating model discovery and download from Hugging Face Hub. It supports CPU, Apple Silicon, CUDA, ROCm, and Intel GPUs, allowing you to execute GGUF files directly on your hardware platform.

How do I select the right GGUF file for resource-constrained devices?▼

Select the right GGUF file by comparing model size, precision, and hardware compatibility. This Skill helps you quickly identify a llama.cpp model that fits your memory constraints while still delivering acceptable performance.

Can I discover and download llama.cpp models from Hugging Face Hub?▼

Yes, you can discover and download llama.cpp models from Hugging Face Hub. This Skill automates the search process, streamlining the acquisition of GGUF files for local execution across various hardware platforms.

Does local inference with llama.cpp support Apple Silicon and CUDA?▼

Local inference with llama.cpp supports Apple Silicon and CUDA, alongside ROCm, Intel GPUs, and CPU configurations. This broad hardware compatibility ensures you can run GGUF files efficiently on your preferred platform.

Do I need llama.cpp to perform local inference and model selection?▼

Yes, you need llama.cpp to perform local inference and model selection. This Skill requires the llama-cpp-python dependency to automate the discovery, download, and execution of GGUF models on your local machine.

Related Skills

Explore More Product & Management Skills→
AlexiosBluffMaraAlexiosBluffMara
2

llama-cpp

Run local GGUF inference and discover models via Hugging Face Hub.

Community
Intermediate
tdiminotdimino
41

llama-cpp

Run GGUF model inference with llama.cpp and apply LoRA adapters.

Community
Advanced
SriRamkunamsettySriRamkunamsetty

llama-cpp

Run local GGUF inference and discover Hugging Face models for llama.cpp.

Community
Intermediate
zengbaochengzengbaocheng

llama-cpp

Run local GGUF inference with llama.cpp using llama-server or llama-cli.

Community
Advanced