skills.rest
hungthinh04
hungthinh04COMMUNITY
Explore all skills by this author →
GitHub

llama-cpp

Discover GGUF models and generate llama.cpp inference commands for Hugging Face repos.

Updated May 12, 2026
Search Tags:#quantization#model discovery#gguf#llama.cpp#local inference#hugging face#gpu offload
One-click install
npx skills add https://github.com/hungthinh04/Hermes_AI_Agent --skill llama-cpp-hungthinh04
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/hungthinh04/Hermes_AI_Agent/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/hungthinh04/Hermes_AI_Agent --skill llama-cpp-hungthinh04

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill removes the guesswork from running Hugging Face models with llama.cpp by helping you find compatible GGUF files, choose a sensible quantization, and launch local inference quickly.

Core Features & Use Cases

  • Model Discovery: Locate llama.cpp-ready repositories and inspect available GGUF files and sizes.
  • Quant Selection: Choose the best quantization for your RAM, VRAM, and quality needs.
  • Local Inference: Generate exact llama-server or llama-cli commands for CPU, Apple Silicon, CUDA, ROCm, and Intel GPU setups.
  • Use Case: If you want to serve a model from Hugging Face on your laptop or workstation, this Skill helps you identify the right file and run it with minimal setup friction.

Quick Start

Ask the Skill to find the best GGUF file for a Hugging Face model repo and generate the exact llama.cpp command for your hardware.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find the right GGUF file for a Hugging Face model to run locally?▼

To find the right GGUF file for local inference, inspect the Hugging Face repository tree API to locate available llama.cpp-ready GGUF files and match their sizes to your hardware RAM and VRAM constraints.

How do I build llama.cpp commands for Apple Silicon or CUDA GPU offload?▼

Building llama.cpp commands for Apple Silicon, CUDA, or ROCm involves hardware-aware flag selection to enable GPU offload. The Skill generates exact llama-server or llama-cli commands tailored to your specific device deployment.

What is the best quantization to choose for local inference on my workstation?▼

The best quantization for local inference depends on your RAM, VRAM, and quality needs. Model-size matching evaluates your hardware to recommend a sensible GGUF quantization that balances memory limits and output fidelity.

Does llama.cpp support multimodal projector handling for local models?▼

Yes, llama.cpp supports multimodal projector handling for local models. The Skill ensures exact repository inspection to identify multimodal projector files and reconstructs the correct launch commands for these setups.

Can I generate inference commands for Intel GPU deployments using Hugging Face repos?▼

Yes, you can generate inference commands for Intel GPU deployments. The Skill reconstructs exact llama.cpp commands from Hugging Face local-app pages or tree API filenames for CPU, Apple Silicon, CUDA, ROCm, and Intel GPU setups.

Related Skills

Explore More Software Engineering Skills→
whichguywhichguy

llama-cpp

Run llama.cpp GGUF model inference locally on CPU, Apple Silicon, CUDA, ROCm, or Intel GPUs.

Community
Advanced
zengbaochengzengbaocheng

llama-cpp

Run local GGUF inference with llama.cpp using llama-server or llama-cli.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
mahg-esmahg-es
3

llm-local-deploy

Deploy local LLMs with llama.cpp, Ollama, or vLLM via docker-compose.

Official
Intermediate