skills.rest
iani-kuli
iani-kuliCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

huggingface-local-models

Discover and run GGUF models from Hugging Face Hub with llama.cpp.

Updated May 5, 2026
Search Tags:#llm#quantization#inference#huggingface#gguf#llama.cpp
One-click install
npx skills add https://github.com/iani-kuli/harness_bro --skill huggingface-local-models-iani-kuli
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: huggingface-local-models
Source: https://github.com/iani-kuli/harness_bro/tree/main/.claude/skills/ported/huggingface-local-models
Command: npx skills add https://github.com/iani-kuli/harness_bro --skill huggingface-local-models-iani-kuli

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This skill simplifies the complex process of discovering, selecting, and running GGUF-formatted models from Hugging Face locally, removing the guesswork from hardware compatibility and quantization.

Core Features & Use Cases

  • Hub-First Discovery: Directly search and identify llama.cpp-compatible models using Hugging Face URL parameters.
  • Quantization Guidance: Automatically determine the optimal quantization level based on hardware constraints and model requirements.
  • Local Serving: Launch models via llama-cli or llama-server with precise configuration for Metal, CUDA, or ROCm acceleration.

Quick Start

Use the huggingface-local-models skill to search for and launch a Qwen model with Q4_K_M quantization for local inference.

Frequently Asked Questions about huggingface-local-models

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run Hugging Face GGUF models locally using llama.cpp?▼

Quantization reduces model size and memory requirements; this skill automatically determines the optimal quantization level, such as Q4_K_M, based on your specific hardware constraints and the selected model's requirements.

Does local LLM inference with llama.cpp support Apple Silicon, NVIDIA, and AMD acceleration?▼

You can discover llama.cpp-compatible models by searching directly through Hugging Face URL parameters, allowing targeted identification of GGUF-formatted models suited for your local deployment needs.

What's the best way to configure llama-server for local Hugging Face model deployment?▼

The best way to configure local serving is by launching models via llama-server with precise acceleration settings, leveraging automated hardware detection to optimize inference across your specific compute environment.

Why does quantization matter when running GGUF models locally?▼

Quantization matters because it balances model performance with hardware limitations, and this skill removes the guesswork by automatically selecting the appropriate quantization level to fit your system's available resources.

Related Skills

Explore More Software Engineering Skills→
vikrant-projectvikrant-project
2

llama-cpp

Execute GGUF inference locally with llama.cpp and Python bindings.

Community
Intermediate
rawvegrawveg
40

ollama

Run and interact with local AI models via the Ollama API.

Community
Advanced
georgekhananaevgeorgekhananaev
27

prompt-compressor

Compress verbose prompts and context to reduce token usage before LLM processing.

Community
Basic
ldzhhxxldzhhxx
1

modal-serverless-gpu

Deploy machine learning models as scalable APIs on serverless GPU cloud.

Community
Advanced