skills.rest
gke-labs
gke-labsOFFICIAL
Explore all skills by this author →
GitHub

gke-inference-quickstart

Generate optimized Kubernetes manifests for AI inference on Google Kubernetes Engine.

42|32|Updated May 7, 2026
Search Tags:#deployment#ai#kubernetes#gcloud#inference#gke#ml
One-click install
npx skills add https://github.com/gke-labs/kube-agents --skill gke-inference-quickstart-gke-labs
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: gke-inference-quickstart
Source: https://github.com/gke-labs/kube-agents/tree/main/agents/platform/skills/gke-inference-quickstart
Command: npx skills add https://github.com/gke-labs/kube-agents --skill gke-inference-quickstart-gke-labs

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill simplifies the complex process of configuring and deploying AI/ML inference workloads on GKE by automating manifest generation and ensuring compatibility between models, servers, and hardware.

Core Features & Use Cases

  • Automated Manifest Generation: Uses Google's best practices to create optimized Kubernetes manifests for inference.
  • Hardware Discovery: Identifies valid combinations of models, inference servers, and GPU/TPU accelerators.
  • Use Case: Quickly deploy a Llama or Gemma model to a GKE cluster by generating a validated configuration that matches your specific hardware and latency requirements.

Quick Start

Use the gke-inference-quickstart skill to generate a deployment manifest for the gemma-2-9b-it model using the vllm server and nvidia-l4 accelerators.

Frequently Asked Questions about gke-inference-quickstart

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy AI models on GKE?▼

Deploy AI models on GKE by automating Kubernetes manifest generation, which validates compatibility between your model, inference server, and hardware accelerators to ensure high-performance serving.

How do I generate Kubernetes manifests for ML inference?▼

Generate Kubernetes manifests for ML inference by validating combinations of models, servers, and hardware accelerators, applying Google's best practices to produce optimized configurations for your specific latency requirements.

Can I deploy a Llama or Gemma model on GKE with vllm?▼

Yes, you can deploy models like Gemma or Llama on GKE using the vllm server, validating the configuration against specific accelerators like nvidia-l4 to match your hardware and latency requirements.

What's the best way to match AI inference servers with GPU accelerators on Kubernetes?▼

Match AI inference servers with GPU accelerators on Kubernetes by using hardware discovery tools that identify valid combinations of models, servers, and TPUs or GPUs for infrastructure-aware workload configuration.

Does deploying ML workloads on Google Kubernetes Engine require hardware compatibility checks?▼

Yes, deploying ML workloads on Google Kubernetes Engine requires hardware compatibility checks to ensure the selected model, inference server, and GPU or TPU accelerator combinations are valid and optimized for serving.

Related Skills

Explore More Software Engineering Skills→
googlegoogle
17.1k

gke-inference

Automate AI/ML inference deployment and optimization on Google Kubernetes Engine.

Official
Advanced
tadod12tadod12

serving-llms-vllm

Serve LLM inference via OpenAI-compatible endpoints using vLLM.

Community
Advanced
Sir-chawakornSir-chawakorn
33

containerization

Create Dockerfiles, docker-compose configurations, and Kubernetes deployment manifests.

Community
Intermediate
1thirteeng31thirteeng3

modal-serverless-gpu

Deploy ML workloads to Modal's serverless GPU cloud with auto-scaling.

Community
Advanced