skills.rest
tangzheng202202
tangzheng202202COMMUNITY
Explore all skills by this author →
GitHub

local-llm-deployment

Automate hardware detection and generate local LLM deployment scripts.

1|Updated Apr 13, 2026
Search Tags:#deploy#ollama#local-llm#llama.cpp#hardware-detection#model-quantization
One-click install
npx skills add https://github.com/tangzheng202202/hermes-skills --skill local-llm-deployment
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: local-llm-deployment
Source: https://github.com/tangzheng202202/hermes-skills/tree/main/03-mlops/mlops/local-llm-deployment
Command: npx skills add https://github.com/tangzheng202202/hermes-skills --skill local-llm-deployment

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Deploy local LLMs optimized for hardware, reducing setup time and trial-and-error when choosing and configuring models.

Core Features & Use Cases

  • Hardware detection and feasibility assessment to select model size and quantization
  • Script generation for deployment on Ollama, llama.cpp, or vLLM
  • Troubleshooting guidance and API integration tips for local serving

Quick Start

Tell me how to deploy a local LLM on my machine by detecting hardware and generating deployment scripts for Ollama or llama.cpp.

Frequently Asked Questions about local-llm-deployment

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy a local LLM optimized for my hardware?▼

To deploy a local LLM optimized for your hardware, you can automate hardware detection to assess system feasibility and generate deployment scripts for Ollama, llama.cpp, or vLLM based on your RAM and GPU.

What is the best way to choose a local LLM model size and quantization for my GPU?▼

Choosing the right local LLM model size and quantization involves auto-detecting your system specifications, performing a feasibility assessment, and mapping your available RAM and GPU capacity to the correct model configuration.

Does local LLM deployment with llama.cpp work on macOS, Linux, and Windows?▼

Yes, local LLM deployment with llama.cpp works across Linux, macOS, and Windows environments, allowing personal developers to deploy models like Llama, Gemma, or Qwen locally with various hardware configurations.

Ollama vs vLLM: which should I choose for local LLM serving?▼

When choosing between Ollama and vLLM for local LLM serving, this process generates cross-tool deployment scripts tailored to your hardware, ensuring both options are configured correctly based on your specific system feasibility.

How do I test the API after deploying a local LLM?▼

To test the API after deploying a local LLM, you can follow basic API testing guidance and troubleshooting tips provided alongside the generated deployment scripts to ensure your local serving setup is functioning correctly.

Related Skills

Explore More Software Engineering Skills→
tangzheng202202tangzheng202202
1

local-llm-deployment-assessment

Assess local hardware and generate optimized LLM deployment configurations.

Community
Advanced
RachasumanthRachasumanth

inference-deploy

Deploy trained machine learning models for inference using vLLM, TGI, Ollama, and llama.cpp.

Community
Advanced
HuskyDannyHuskyDanny

unsloth

Fine-tune LLMs with LoRA and QLoRA using the Unsloth framework.

Community
Advanced
ovachieverovachiever
52

openrlhf-training

Train large language models with RLHF using Ray and vLLM acceleration.

Community
Advanced