What problem does it solve?
Moving an existing AI inference workload from Cloud Run, the Gemini API, or Gemini Enterprise Agent Platform to self-hosted inference on Google Kubernetes Engine involves hardware sizing, model staging, manifest authoring, and traffic cutover decisions that are easy to get wrong. This Skill guides an agent through a structured 4-phase migration workflow so nothing is missed.
Core Features & Use Cases
- Guided 4-Phase Workflow: Discovery (inspect existing infrastructure with gcloud), Solution Design (VRAM sizing, accelerator and storage selection, manifest generation), Implementation (provisioning, model staging via cluster Jobs, kubectl deployment), and Validation and Cutover (health checks, inference tests, rollback readiness).
- Opinionated Golden Path: Defaults to vLLM serving, Custom Compute Classes for accelerator obtainability, Cloud Storage FUSE or Managed Lustre for model weights, GKE Gateway API for routing, Workload Identity for security, and Managed Service for Prometheus for observability.
- Deterministic VRAM Sizing: Calculates GPU memory requirements from parameter count, quantization, and KV cache overhead with a 20% safety margin to recommend the right accelerator (L4, A100, H100).
- Use Case: A team running a Gemma 2 9B model on Cloud Run wants to cut inference costs by self-hosting on GKE. The Skill inspects the Cloud Run service, sizes an L4 node pool, stages weights to a GCS-backed PVC, generates vLLM and Gateway manifests, and walks through validation and traffic cutover.
Quick Start
Ask the agent to migrate my existing Cloud Run AI inference service to self-hosted vLLM inference on GKE.
