skills.rest
tangzheng202202
tangzheng202202COMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Execute CPU-based LLM inference with GGUF quantization on Apple Silicon and non-NVIDIA GPUs.

1|Updated Apr 13, 2026
Search Tags:#quantization#cpu#gguf#apple-silicon#llama.cpp#edge-deployment
One-click install
npx skills add https://github.com/tangzheng202202/hermes-skills --skill llama-cpp-tangzheng202202
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/tangzheng202202/hermes-skills/tree/main/03-mlops/mlops/inference/llama-cpp
Command: npx skills add https://github.com/tangzheng202202/hermes-skills --skill llama-cpp-tangzheng202202

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Pure C/C++ LLM inference with minimal dependencies, optimized for CPUs and non-NVIDIA hardware.

Core Features & Use Cases

  • CPU/Apple Silicon optimized inference for on-device and edge deployments without CUDA.
  • GGUF quantization support for reduced memory and faster performance on CPU and compatible GPUs.
  • Flexible hardware acceleration including Metal (Apple Silicon), ROCm (AMD), and CPU backends for broad compatibility.

Quick Start

Install llama.cpp and run a basic inference on a supported model.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run LLM inference on Apple Silicon or non-NVIDIA hardware without CUDA?▼

You can run CPU-based LLM inference on Apple Silicon or non-NVIDIA hardware using llama.cpp, which achieves on-device deployment without CUDA dependencies.

Can I use GGUF quantization models for edge deployment on CPUs?▼

Yes, GGUF quantization formats are supported to reduce memory usage and accelerate LLM performance on CPUs and compatible GPUs for edge deployment scenarios.

Does llama.cpp support hardware acceleration for AMD GPUs and Macs?▼

llama.cpp supports flexible hardware acceleration including Metal for Apple Silicon Macs, ROCm for AMD GPUs, and standard CPU backends for broad compatibility.

What is the best way to deploy LLMs on edge devices with minimal dependencies?▼

The best way to deploy LLMs on edge devices with minimal dependencies is using pure C/C++ inference, which avoids heavy framework requirements and optimizes CPU-first execution.

When should I choose CPU-first LLM inference over GPU-accelerated alternatives?▼

You should choose CPU-first LLM inference when operating in edge scenarios on Apple Silicon Macs or non-NVIDIA hardware where CUDA is unavailable and GGUF quantization is preferred.

Related Skills

Explore More Software Engineering Skills→
SignmanalSignmanal
1

llama-cpp

Run local GGUF model inference with llama.cpp across CPU and GPU hardware.

Community
Intermediate
ovachieverovachiever
52

llamaguard

Classify user input and system outputs across six safety categories.

Community
Advanced
GitnappGitnapp

gguf-quantization

Convert Hugging Face models into GGUF files with Q4_K_M quantization.

Community
Advanced
gagan114662gagan114662

llama-cpp

Run LLM inference on CPUs, Apple Silicon, and non-NVIDIA GPUs using GGUF quantization.

Community
Intermediate