ecs-recovery

Restarts Docker and ECS on EC2-backed ECS nodes with NAT verification and optional draining.

Updated Apr 14, 2026
One-click install
npx skills add https://github.com/amitkarpe/agent-skills --skill ecs-recovery-amitkarpe
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ecs-recovery
Source: https://github.com/amitkarpe/agent-skills/tree/main/skills/ecs-recovery
Command: npx skills add https://github.com/amitkarpe/agent-skills --skill ecs-recovery-amitkarpe

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? When ECS tasks get stuck in PENDING or ACTIVATING, Docker NAT breaks, or awslogs fails on specific nodes, operators need a controlled, evidence-backed way to recover one node at a time without destabilizing the whole cluster. ## Core Features & Use Cases - Single-node restart with verification: Restarts docker and ecs via SSM on one EC2 instance, then confirms docker=active, ecs=active, and that the Docker NAT chain returned. - Optional drain before restart: Sets a container instance to DRAINING when the caller has ecs:UpdateContainerInstancesState, and treats missing permission as a hard stop. - Durable evidence capture: Writes before/after command output and a summary to a timestamped directory under ~/.AGENTS-temp for audit and review. - Use Case: A service is flapping because one ECS node's Docker NAT chain is missing. Resolve the container instance ARN from the EC2 ID, drain it if permitted, restart Docker and ECS, and verify NAT and service counts before moving on. ## Quick Start Ask the agent to restart Docker and ECS on EC2 instance i-0123456789abcdef0 in cluster production using your AWS profile and region, then verify the Docker NAT chain returned.

Frequently Asked Questions about ecs-recovery

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I safely restart Docker and ECS on one ECS node?

Run scripts/restart-docker-ecs-one.sh with the EC2 instance ID, AWS profile, and region. It captures before state via SSM, restarts docker then ecs, and verifies docker=active, ecs=active, and the Docker NAT chain afterward.

How do I drain an ECS container instance before recovery?

Use scripts/drain-container-instance.sh with the cluster, profile, region, and EC2 instance ID. It resolves the container instance ARN and calls ecs update-container-instances-state with status DRAINING, which requires the ecs:UpdateContainerInstancesState permission.

Does this ECS recovery workflow work on Fargate?

No, it only applies to EC2-backed ECS clusters because it restarts the docker and ecs system services on the host via SSM. Fargate has no accessible node, so this workflow explicitly excludes Fargate-only environments.

What happens if I lack ecs:UpdateContainerInstancesState permission?

The drain step is treated as a real blocker and the workflow stops rather than faking drain behavior. You can still restart one node at a time without draining, but you should not proceed as if the node were drained.

When should I stop an ECS node recovery instead of continuing?

Stop immediately if a critical service stays degraded, the Docker NAT chain does not return, or Docker or ECS fails to come back active. Also stop if service counts worsen and do not recover promptly after a restart.