skills.rest
aws
awsOFFICIAL
Explore all skills by this author →
WebsiteGitHub

migrate-to-msk

Migrates self-managed Apache Kafka workloads to Amazon MSK Express through discovery, assessment, and simulation phases.

2.5k|282|Updated Apr 23, 2026
Search Tags:#AWS#Streaming#Kafka#Migration#amazon msk#compatibility assessment#msk express
One-click install
npx skills add https://github.com/aws/agent-toolkit-for-aws --skill migrate-to-msk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: migrate-to-msk
Source: https://github.com/aws/agent-toolkit-for-aws/tree/main/skills/specialized-skills/analytics-skills/migrate-to-msk
Command: npx skills add https://github.com/aws/agent-toolkit-for-aws --skill migrate-to-msk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

Moving a self-managed Apache Kafka cluster to Amazon MSK Express requires inventorying the source cluster, validating compatibility across topology, Kafka version, configs, auth, and quotas, and sizing a cost-effective target — a manual process prone to missed incompatibilities and wrong broker counts.

Core Features & Use Cases

  • Discovery: Inventories the source cluster from IaC files (Terraform, CDK, Docker Compose, Kubernetes manifests), Kafka CLI output, or manual input into a structured cluster-config.json.
  • Assessment: Runs a five-pillar compatibility check (topology, Kafka version, configs, auth, quotas) via scripts/compatibility.py and produces a target Express specification with instance type, broker count, and monthly cost using the managing-amazon-msk sizing logic.
  • Simulation: Optionally provisions a trial MSK Express cluster with load-testing infrastructure to validate performance on your real workload before committing.
  • Use Case: A platform team running Kafka on Kubernetes asks the agent to analyze their manifests, receives a compatibility verdict with ACTION_REQUIRED items flagged, a right-sized Express recommendation with cost projection, and guidance on MSK Replicator for data migration.

Quick Start

Ask the agent to assess your Kafka cluster for migration to MSK Express by pointing it at your infrastructure files or an existing cluster-config.json.

Frequently Asked Questions about migrate-to-msk

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I migrate a self-managed Kafka cluster to Amazon MSK?▼

Run the three-phase flow: discovery inventories your cluster from IaC files or Kafka CLI output into cluster-config.json, assessment validates MSK Express compatibility and produces a sized target with cost, and an optional simulation load-tests Express against your workload. Data migration uses MSK Replicator.

What Kafka versions does MSK Express support?▼

MSK Express supports Apache Kafka versions 3.6, 3.8, and 3.9. Other versions trigger an advisory to validate client compatibility, and sources older than 2.8.1 cannot use MSK Replicator and need a MirrorMaker 2 based solution for data migration.

Can I migrate from MSK Standard to MSK Express with this skill?▼

No. The skill covers only migrations from self-managed Apache Kafka (on-premises, EC2, Docker, Kubernetes, or other non-MSK deployments) to MSK Express. MSK Standard to Express migrations are explicitly out of scope.

Does the compatibility assessment make AWS API calls?▼

No. The compatibility script is a pure file processor run locally via uv run with standard-library-only dependencies. It reads cluster-config.json and emits a JSON verdict without network access or live cluster calls.

What authentication mechanisms work with MSK Express?▼

MSK Express supports unauthenticated access, TLS, SASL/SCRAM, and IAM authentication, with TLS in transit required for all authenticated mechanisms. Custom SASL/OAUTHBEARER providers and mechanisms like Kerberos or SASL/PLAIN are flagged as ACTION_REQUIRED since Express does not support them.

Why does the assessment report fewer brokers than my source cluster?▼

Source clusters are often over-provisioned. The sizing logic models peak throughput, one-AZ-down headroom, and partition limits, and Express brokers handle up to 3x ingress per broker, so a smaller right-sized target does not reduce resilience.

Related Skills

Explore More Software Engineering Skills→
api7api7
1

a6-plugin-kafka-logger

Push APISIX request and response logs to Kafka topics via the a6 kafka-logger plugin.

Official
Advanced
awsaws
2.3k

managing-amazon-msk

Automate Amazon MSK cluster patching, upgrades, and monitoring across multi-AZ deployments.

Official
Advanced
AndreKuraitAndreKurait

managing-amazon-msk

Operate Amazon MSK Provisioned clusters with patching, upgrade, and restart guardrails.

Community
Advanced
armoin2018armoin2018
4

ailey-data-kafka

Automate Kafka workflows for real-time data pipelines across Confluent Cloud, Platform, Apache Kafka, and AWS MSK.

Community
Advanced