skills.rest
zhouboyu-xreal
zhouboyu-xrealCOMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Discover, deploy, and run llama.cpp models from Hugging Face Hub.

Updated Jun 9, 2026
Search Tags:#deployment#inference#model discovery#AI model#Hugging Face#llama.cpp
One-click install
npx skills add https://github.com/zhouboyu-xreal/Hermes-Memory --skill llama-cpp-zhouboyu-xreal
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/zhouboyu-xreal/Hermes-Memory/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/zhouboyu-xreal/Hermes-Memory --skill llama-cpp-zhouboyu-xreal

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires llama-cpp-python, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill solves the challenge of finding, deploying, and running AI models on various hardware and platforms, simplifying the process for users with varying levels of technical expertise.

Core Features & Use Cases

  • Model Discovery: Search and discover AI models compatible with llama.cpp on Hugging Face Hub.
  • Deployment: Build and launch models directly from the Hub for both CPU and GPU inference.
  • Inference: Execute various models locally on systems equipped with the required hardware.

Quick Start

Install the required software with:

pip install llama.cpp-python

Then run:

llama-cli -hf <model-name>:<quant>

<model-name> should be replaced with the specific Hugging Face Hub repository path to the model.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy and run Hugging Face models locally using llama.cpp?▼

To deploy Hugging Face models locally, install the llama-cpp-python dependency and execute the llama-cli command with the specific Hugging Face Hub repository path and quantization suffix to launch inference directly on your hardware.

Can I run local inference on Apple Silicon or CPU without a dedicated GPU?▼

Yes, local inference is supported across various hardware configurations, including standard CPU environments and Apple Silicon, allowing flexible AI model execution without requiring a dedicated GPU.

How do I discover AI models compatible with llama.cpp on Hugging Face Hub?▼

Model discovery is integrated directly with the Hugging Face Hub, allowing you to search and identify AI models specifically optimized for llama.cpp before building and launching them for local inference.

Do I need specific dependencies to build and launch models from Hugging Face Hub?▼

Yes, you need to install the llama-cpp-python package via pip to build and launch models from Hugging Face Hub, which enables the local app functionality and direct command execution.

What is the best way to streamline AI model deployment and inference for different hardware?▼

Streamlining AI model deployment involves using Hugging Face Hub integration to discover models, then executing direct commands to build and run them locally across CPU, Apple Silicon, or GPU hardware environments.

Why use llama.cpp for local inference instead of other local deployment options?▼

Using llama.cpp simplifies finding and running AI models by focusing on optimized local execution across varied hardware, leveraging Hugging Face Hub's local app functionality for direct command execution.

Related Skills

Explore More Software Engineering Skills→
CyapstayeCyapstaye
9

llama-cpp

Discover GGUF models on Hugging Face and run local inference with llama.cpp.

Community
Advanced
datathingsdatathings
10

llamacpp

Reference the llama.cpp C API for model loading and inference workflows.

Official
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
hochoa13hochoa13

llama-cpp

Run GGUF quantized LLM inference on CPUs, Apple Silicon, and consumer GPUs.

Community
Intermediate