skills.rest
pluginagentmarketplace
pluginagentmarketplaceCOMMUNITY
Explore all skills by this author →
GitHub

model-deployment

Deploy LLMs to production with vLLM, TGI, and Ollama configurations.

2|1|Updated Nov 18, 2025
Search Tags:#kubernetes#inference#ollama#vllm#model serving#llm deployment#tgi
One-click install
npx skills add https://github.com/pluginagentmarketplace/custom-plugin-ai-engineer --skill model-deployment-pluginagentmarketplace
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-deployment
Source: https://github.com/pluginagentmarketplace/custom-plugin-ai-engineer/tree/main/skills/model-deployment
Command: npx skills add https://github.com/pluginagentmarketplace/custom-plugin-ai-engineer --skill model-deployment-pluginagentmarketplace

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill addresses the complexities of deploying Large Language Models (LLMs) into production environments, offering strategies and tools to ensure performance, scalability, and cost-effectiveness.

Core Features & Use Cases

  • Diverse Deployment Options: Explores and provides examples for various serving frameworks like vLLM, Text Generation Inference (TGI), and Ollama.
  • Optimization Techniques: Details methods such as quantization (4-bit, 8-bit) and batching to reduce resource consumption and improve inference speed.
  • Infrastructure as Code: Includes Dockerfiles, Docker Compose configurations, and Kubernetes manifests for streamlined deployment.
  • Monitoring & Best Practices: Outlines essential metrics for monitoring and provides a checklist for secure and efficient LLM deployment.
  • Use Case: A machine learning engineer needs to deploy a fine-tuned Llama-2 model for a customer-facing chatbot. They can use this Skill to choose between vLLM for low latency or TGI for Hugging Face ecosystem integration, and generate the necessary Docker and Kubernetes configurations.

Quick Start

Use the model-deployment skill to deploy the Llama-2-7b-chat-hf model using vLLM.

Frequently Asked Questions about model-deployment

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What is the best way to deploy an LLM to production?▼

Deploying an LLM to production is best handled using serving frameworks like vLLM, TGI, or Ollama, combined with Docker and Kubernetes infrastructure as code to ensure performance, scalability, and cost-effectiveness.

How do I deploy Llama-2 using vLLM and Docker?▼

You can deploy Llama-2 using vLLM by utilizing provided Dockerfiles, Docker Compose configurations, and Kubernetes manifests, which streamline the containerization and orchestration process for serving inference.

Which serving framework should I choose for LLM deployment?▼

Choosing an LLM serving framework depends on your needs: vLLM offers low latency, TGI provides Hugging Face ecosystem integration, and Ollama offers alternative deployment options for production environments.

How does quantization improve LLM inference speed?▼

Quantization improves LLM inference speed and reduces resource consumption by optimizing model weights to 4-bit or 8-bit formats, while batching processes multiple requests together for better throughput.

Can I use Kubernetes to scale my LLM inference workloads?▼

Yes, you can use Kubernetes to scale LLM inference workloads by applying included infrastructure as code examples like Kubernetes manifests, ensuring robust deployment alongside monitoring and security best practices.

Related Skills

Explore More Software Engineering Skills→
SriRamkunamsettySriRamkunamsetty

serving-llms-vllm

Serve OpenAI-compatible LLM endpoints with vLLM using quantization and tensor parallelism.

Community
Advanced
dallaydallay
1

spring-boot-actuator

Configure Spring Boot Actuator endpoints for monitoring, health checks, and metrics export.

Official
Advanced
fakhriaditiarahmanfakhriaditiarahman
1

llm-ops-engineer

Deploy, fine-tune, and monitor LLMs with vLLM, TGI, and Ollama.

Community
Advanced
hotriluanhotriluan

ck:devops

Deploy cloud infrastructure across Cloudflare, Docker, Google Cloud, and Kubernetes.

Community
Advanced