managing-amazon-msk

Diagnose, size, monitor, and maintain Amazon MSK Provisioned Standard and Express clusters.

Updated Jul 21, 2026
Ready to run
npx skills add https://github.com/richardnroman/Synthetic-Data-Exchange-Licensing-Platform --skill managing-amazon-msk-richardnroman
💬 Prompt for ChatGPT / Claude Chat▼
Please help me install this Agent Skill.
Skill: managing-amazon-msk
Source: https://github.com/richardnroman/Synthetic-Data-Exchange-Licensing-Platform/tree/main/.agents/skills/managing-amazon-msk
Command: npx skills add https://github.com/richardnroman/Synthetic-Data-Exchange-Licensing-Platform --skill managing-amazon-msk-richardnroman

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Operating Amazon MSK Provisioned clusters requires distinguishing between Standard and Express broker behaviors that training data routinely conflates, leading to wrong advice on metrics, patching, storage, and sizing. This Skill provides verified operational guidance for performance troubleshooting, consumer lag diagnosis, storage management, cluster sizing, client tuning, CloudWatch monitoring, and maintenance operations. ## Core Features & Use Cases - Performance and Lag Troubleshooting: Diagnose high CPU, traffic shaping, consumer lag, and rebalance storms using the correct CloudWatch metrics for each broker type. - Deterministic Cluster Sizing: Run the bundled msk_sizing.py script to compute broker counts, instance choices, bottlenecks, and monthly costs for Standard and Express clusters. - Monitoring and Maintenance: Create CloudWatch alarms and dashboards, manage EBS and tiered storage, and handle patching, version upgrades, and rolling restarts safely. - Use Case: A user reports consumer lag growing on an MSK cluster. The Skill first identifies the broker type via describe-cluster-v2, then walks through broker-side versus client-side diagnosis using the correct metrics for that type. ## Quick Start Ask the assistant to diagnose why your MSK cluster has high CPU or to size a new MSK cluster from your throughput, partition, and retention requirements.

Frequently Asked Questions about managing-amazon-msk

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I choose between MSK Standard and Express brokers?▼

Choose Express for most workloads since it offers fully managed storage, no maintenance windows, and up to 3x more throughput per broker. Choose Standard only when you need custom broker configurations like replication factor other than 3 or a 2-AZ deployment.

How do I size an Amazon MSK cluster?▼

Run the bundled msk_sizing.py script with your average and peak ingress and egress, partition count, retention hours, and replication factor. It returns broker counts, bottleneck analysis, and monthly costs per instance type, plus a recommended pick per class.

Why is my MSK cluster showing high CPU?▼

The most common cause is client-side batching misconfiguration, specifically linger.ms set to 0, which sends one request per message and saturates broker request handlers. Check RequestHandlerAvgIdlePercent and client batch.size and linger.ms before scaling brokers.

Does MSK Express emit UnderReplicatedPartitions?▼

No, Express brokers do not emit UnderReplicatedPartitions, OfflinePartitionsCount, or KafkaDataLogsDiskUsed. Monitor ProduceThrottleTime, FetchThrottleTime, StorageUsed, and consumer lag instead when operating Express clusters.

What happens during MSK maintenance and rolling restarts?▼

Standard clusters enter MAINTENANCE state and restart brokers one at a time, causing brief client reconnects and temporary URP spikes. Express clusters stay ACTIVE with time-distributed single broker reboots. Never restart additional brokers or reassign partitions while URP is elevated.

When should I not use this MSK skill?▼

Do not use it for MSK Connect, MSK Serverless, or MSK Replicator tasks, as it covers only MSK Provisioned clusters with Standard and Express broker types. Those services have different metrics, limits, and operational models.