skills.rest
BagelHole
BagelHoleCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

model-serving-kubernetes

Deploy and manage ML models on Kubernetes with KServe and Triton.

46|4|Updated Jan 27, 2026
Search Tags:#mlops#llm#kubernetes#gpu#triton#model serving#kserve
One-click install
npx skills add https://github.com/BagelHole/DevOps-Security-Agent-Skills --skill model-serving-kubernetes
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-serving-kubernetes
Source: https://github.com/BagelHole/DevOps-Security-Agent-Skills/tree/main/devops/orchestration/model-serving-kubernetes
Command: npx skills add https://github.com/BagelHole/DevOps-Security-Agent-Skills --skill model-serving-kubernetes

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the deployment and management of machine learning models on Kubernetes, enabling scalable, production-ready inference.

Core Features & Use Cases

  • Model Deployment: Deploy various ML models (scikit-learn, PyTorch, TensorFlow, ONNX, LLMs) using KServe or Triton.
  • Advanced Features: Supports canary deployments, A/B testing, autoscaling (CPU/GPU), and model versioning.
  • Use Case: Deploy a new version of a recommendation model using a canary release, gradually shifting 20% of traffic to test its performance before a full rollout.

Quick Start

Install KServe using Helm by adding its repository and performing an install command.

Frequently Asked Questions about model-serving-kubernetes

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy machine learning models on Kubernetes for production inference?▼

Deploy machine learning models on Kubernetes using KServe or NVIDIA Triton Inference Server to achieve scalable, production-ready inference with autoscaling and GPU resource management.

Can I do canary deployments and A/B testing for ML models on Kubernetes?▼

Yes, canary deployments and A/B testing are natively supported for ML models on Kubernetes, allowing you to gradually shift traffic to new model versions before a full rollout.

Do I need the NVIDIA GPU Operator to serve LLMs on Kubernetes?▼

Yes, serving LLMs and managing GPU resources on Kubernetes requires the NVIDIA GPU Operator alongside KServe, kubectl, helm, and Kubernetes 1.28 or higher.

What's the best way to manage GPU autoscaling for inference servers?▼

Manage GPU autoscaling for inference servers by deploying models through KServe and Triton, which facilitate automatic resource scaling based on CPU and GPU utilization.

Does KServe work with PyTorch, TensorFlow, and ONNX model formats?▼

Yes, KServe works with PyTorch, TensorFlow, and ONNX formats, enabling you to deploy various machine learning models and large language models efficiently.

How does Triton Inference Server handle model versioning on Kubernetes?▼

Triton Inference Server handles model versioning on Kubernetes through KServe integration, allowing simultaneous deployment of multiple model versions for testing and rollback.

Related Skills

Explore More Software Engineering Skills→
hafiznaveedchuhan-ctrlhafiznaveedchuhan-ctrl

deploying-kafka-k8s

Deploy and manage Kafka clusters on Kubernetes with the Strimzi operator.

Community
Advanced
romankovsvromankovsv

seldon-core

Deploy ML models on Kubernetes with Seldon Core v2.

Community
Advanced
BagelHoleBagelHole
46

llm-inference-scaling

Scale LLM inference clusters on Kubernetes using KEDA and GPU metrics.

Community
Advanced
duylinhdang1998duylinhdang1998

ml-infrastructure

Build and deploy ML training pipelines with Docker, Kubernetes, and FastAPI.

Community
Advanced