skills.rest
mysticaltech
mysticaltechCOMMUNITY
Explore all skills by this author →
𝕏@mysticaltechGitHub

upgrade-cluster

Upgrade kube-hetzner clusters through phased Terraform convergence, k3s/RKE2 runtime upgrades, and immutable node replacement.

3.9k|563|Updated Jul 30, 2021
Search Tags:#kubernetes#terraform#Hetzner#etcd#node replacement#cluster upgrade#K3s
One-click install
npx skills add https://github.com/mysticaltech/terraform-hcloud-kube-hetzner --skill upgrade-cluster
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: upgrade-cluster
Source: https://github.com/mysticaltech/terraform-hcloud-kube-hetzner/tree/main/.claude/skills/upgrade-cluster
Command: npx skills add https://github.com/mysticaltech/terraform-hcloud-kube-hetzner --skill upgrade-cluster

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Upgrading a live kube-hetzner Kubernetes cluster on Hetzner Cloud risks accidental infrastructure destruction, etcd quorum loss, and stateful workload downtime. This Skill enforces a phased, evidence-backed upgrade workflow with hard safety rules so module bumps, runtime upgrades, and node replacements happen without destroying live servers or losing data.

Core Features & Use Cases

  • Phased Upgrade Workflow: Separates Terraform module convergence from k3s/RKE2 runtime upgrades, with a protected no-destroy gate that stops any plan deleting hcloud servers, networks, load balancers, or volumes.
  • Immutable Node Replacement: Provides a field-proven reference workflow for replacing compromised, deprecated, or capacity-exhausted hosts while preserving etcd quorum, CSI volumes, and load balancer targets.
  • HA and Safety Guardrails: Covers drain/eviction trade-offs, singleton StatefulSet volume moves, etcd snapshots, firewall open/close proof, and git hygiene rules preventing secret or state file commits.
  • Use Case: An operator needs to move a production cluster from k3s v1.30 to v1.32 and replace deprecated cx21 control planes. The Skill walks through preflight state backup, one-minor-at-a-time runtime upgrades, capacity checks, quorum-safe control-plane replacement, and final firewall closure with concrete proof commands.

Quick Start

Ask the assistant to upgrade your existing kube-hetzner cluster to a newer module version and k3s release, providing your Terraform root path and kubeconfig location.

Frequently Asked Questions about upgrade-cluster

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I upgrade a kube-hetzner cluster without downtime?▼

Upgrade in two phases: first converge the Terraform module with a no-destroy plan gate, then bump k3s or RKE2 one minor version at a time. Verify node readiness, system-upgrade plans, and application health between each step before proceeding.

How do I replace compromised nodes in a Kubernetes cluster?▼

Use immutable node replacement: append fresh nodepools, drain old workers, and replace control-plane voters one at a time while preserving etcd quorum. Take an etcd snapshot first and verify load balancer targets only point to fresh servers afterward.

Can I upgrade k3s multiple minor versions at once?▼

No, upgrade one minor version at a time unless you have explicit upstream proof that skipping minors is safe for that exact version span. Run runtime health checks after each minor before continuing to the next.

Why does my Terraform plan show hcloud servers being destroyed?▼

Unexpected destruction usually means a module refactor changed resource addresses or nodepool list order shifted. Stop immediately, back up state, and use terraform import, state mv, or moved blocks instead of applying the destructive plan.

Does this work with OpenTofu instead of Terraform?▼

Yes, the workflow supports OpenTofu, but you must use the same IaC runner the target root already uses. Substitute tofu for terraform only if the existing cluster root is already managed with OpenTofu.

When should I use drain versus cordon for node upgrades?▼

Use system_upgrade_use_drain = true for replicated stateless workloads, but prefer cordon (drain = false) when draining would move singleton stateful workloads or cause volume attach churn. Handle attached-volume StatefulSets explicitly before draining any node.

Related Skills

Explore More Software Engineering Skills→
kube-dckube-dc
34

manage-cluster

Automate day-2 operations for Kube-DC managed Kubernetes clusters.

Official
Intermediate
AcczdyAcczdy
2

performing-service-account-credential-rotation

Automate service account credential rotation across AD, cloud platforms, and databases.

Community
Advanced
Xipher-LabsXipher-Labs
6

hcloud-cli

Manage Hetzner Cloud infrastructure with the official hcloud CLI.

Official
Advanced
rohitg00rohitg00
944

k8s-backup

Back up and restore Kubernetes workloads using Velero.

Community
Advanced