model-serving-kubernetes

What problem does it solve?

This Skill streamlines the deployment and management of machine learning models on Kubernetes, enabling scalable, production-ready inference.

Core Features & Use Cases

• Model Deployment: Deploy various ML models (scikit-learn, PyTorch, TensorFlow, ONNX, LLMs) using KServe or Triton.
• Advanced Features: Supports canary deployments, A/B testing, autoscaling (CPU/GPU), and model versioning.
• Use Case: Deploy a new version of a recommendation model using a canary release, gradually shifting 20% of traffic to test its performance before a full rollout.

Quick Start

Install KServe using Helm by adding its repository and performing an install command.