managing-amazon-msk

Diagnose, size, monitor, and maintain Amazon MSK Provisioned clusters with Standard and Express brokers.

Updated Jul 1, 2026
One-click install
npx skills add https://github.com/sakicodes/BuildFestHackathon26 --skill managing-amazon-msk-sakicodes
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: managing-amazon-msk
Source: https://github.com/sakicodes/BuildFestHackathon26/tree/main/.agents/skills/managing-amazon-msk
Command: npx skills add https://github.com/sakicodes/BuildFestHackathon26 --skill managing-amazon-msk-sakicodes

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Operating Amazon MSK Provisioned clusters requires distinguishing between Standard and Express broker behaviors that training data routinely conflates, leading to wrong advice on metrics, storage, patching, and sizing. This Skill provides verified operational guidance for performance troubleshooting, consumer lag diagnosis, storage management, cluster sizing, client tuning, CloudWatch monitoring, and maintenance operations. ## Core Features & Use Cases - Performance and Lag Troubleshooting: Diagnose high CPU, latency, traffic shaping, and consumer lag using the correct CloudWatch metrics for each broker type, with critical guardrails like never rebooting brokers while UnderReplicatedPartitions is above zero. - Deterministic Cluster Sizing: Run the bundled msk_sizing.py script to compute broker counts, instance choices, bottlenecks, and monthly costs across all Standard M5/M7g and Express M7g sizes. - Monitoring and Maintenance: Create CloudWatch alarms and dashboards, configure producers and consumers for IAM, SCRAM, or mTLS auth, and handle patching, version upgrades, and rolling restarts safely. - Use Case: A user reports consumer lag growing on an MSK cluster. The Skill first identifies the broker type via describe-cluster-v2, then walks through broker-side versus client-side diagnosis using the correct metrics for that type. ## Quick Start Ask the assistant to diagnose why your MSK cluster has high CPU or growing consumer lag, providing the cluster ARN so it can determine the broker type first.

Frequently Asked Questions about managing-amazon-msk

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I choose between MSK Standard and Express brokers?

Choose Express for most workloads since it offers fully managed storage, no maintenance windows, faster scaling, and up to 3x more throughput per broker. Choose Standard only when you need custom broker configurations like replication factor other than 3 or a 2-AZ deployment.

How do I size an Amazon MSK cluster and estimate monthly cost?

Run the bundled msk_sizing.py script with your average and peak ingress/egress, partition count, retention hours, and replication factor. It returns broker counts, bottlenecks, and per-class cost recommendations across all Standard and Express instance sizes.

Why is my MSK cluster CPU high and how do I fix it?

High MSK CPU is most often caused by client misconfiguration, especially linger.ms set to 0, which sends one request per message and saturates broker request handlers. Check batch.size and linger.ms before scaling brokers, and verify RequestHandlerAvgIdlePercent is above 30%.

Does MSK Express emit UnderReplicatedPartitions metrics?

No, Express brokers do not emit UnderReplicatedPartitions, OfflinePartitionsCount, or KafkaDataLogsDiskUsed. Monitor ProduceThrottleTime, FetchThrottleTime, StorageUsed, and consumer lag instead when operating Express clusters.

What happens during MSK maintenance and rolling broker restarts?

Standard clusters enter MAINTENANCE state and restart brokers one at a time, causing brief client reconnects and temporary URP spikes. Express clusters stay ACTIVE with time-distributed single broker reboots. Topics with RF=1 on Standard become fully unavailable during their broker's restart.

When should I not use this MSK Skill?

Do not use it for MSK Connect, MSK Serverless, or MSK Replicator, as it covers only MSK Provisioned clusters with Standard and Express brokers. Those services have different metrics, APIs, and operational models not addressed here.