skills.rest
Tyler-R-Kendrick
Tyler-R-KendrickCOMMUNITY
Explore all skills by this author →
GitHub

huggingface-local-models

Search Hugging Face Hub and launch GGUF models via llama.cpp.

1|Updated Jul 12, 2026
Search Tags:#llm#quantization#huggingface#gguf#llama.cpp#local-inference
One-click install
npx skills add https://github.com/Tyler-R-Kendrick/slm-training --skill huggingface-local-models-tyler-r-kendrick
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: huggingface-local-models
Source: https://github.com/Tyler-R-Kendrick/slm-training/tree/main/.agents/skills/huggingface-local-models
Command: npx skills add https://github.com/Tyler-R-Kendrick/slm-training --skill huggingface-local-models-tyler-r-kendrick

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This skill removes the complexity of finding, selecting, and configuring local LLMs by providing a standardized workflow for searching the Hugging Face Hub and launching models via llama.cpp.

Core Features & Use Cases

  • Hub Discovery: Search for GGUF-compatible models using specific hardware and parameter constraints.
  • Quantization Selection: Automatically identify the optimal quantization format based on hardware compatibility and memory budgets.
  • Local Serving: Launch OpenAI-compatible local servers or CLI interfaces for immediate model interaction.
  • Use Case: Quickly deploy a Qwen3.6 model on a local machine by selecting the recommended UD-Q4_K_M quantization and starting a local inference server.

Quick Start

Use the huggingface-local-models skill to search for and launch a llama.cpp compatible model from the Hugging Face Hub.

Frequently Asked Questions about huggingface-local-models

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local LLM inference using GGUF models with llama.cpp?▼

Run local LLM inference by discovering GGUF-formatted models on the Hugging Face Hub and executing them via llama.cpp binaries, which supports CPU, Metal, CUDA, and ROCm hardware acceleration.

How do I choose the right quantization format for my local LLM hardware?▼

Choose the right quantization format by evaluating your hardware compatibility and memory budget, allowing the system to automatically identify optimal formats like UD-Q4_K_M for efficient local deployment.

Can I deploy an OpenAI-compatible local server using Hugging Face GGUF models?▼

Yes, you can deploy an OpenAI-compatible local server or CLI interface by launching GGUF model weights via llama.cpp, providing immediate local model interaction and API access.

What hardware acceleration does llama.cpp support for local inference?▼

llama.cpp supports CPU, Metal, CUDA, and ROCm environments for local inference, requiring specific hardware connectivity to the Hugging Face Hub to manage model weights and acceleration.

How do I search for GGUF-compatible models on the Hugging Face Hub for local deployment?▼

Search for GGUF-compatible models on the Hugging Face Hub by applying specific hardware and parameter constraints to find suitable model weights for your local machine setup.

Do I need llama.cpp binaries installed to run Hugging Face models locally?▼

Yes, you need llama.cpp binaries installed to execute local LLM inference, as they handle the model weight execution and hardware-specific acceleration across your computing environment.

Related Skills

Explore More Software Engineering Skills→
DonZzzillaDonZzzilla

llama-cpp

Run local GGUF inference for llama.cpp models with Hugging Face Hub discovery.

Community
Advanced
TehminanazTehminanaz
1

openai-agents-sdk-groq-integration

Integrate the OpenAI Agents SDK with Groq's endpoint in FastAPI applications.

Community
Advanced
RheasilviaRheasilvia

serving-llms-vllm

Deploy OpenAI-compatible LLM inference servers using vLLM with PagedAttention and continuous batching.

Community
Advanced
helix4uhelix4u

llama-cpp

Run LLM inference on CPU, Apple Silicon, and non-NVIDIA GPUs with GGUF models.

Community
Intermediate