processing-s3-uploads-with-step-functions

Enable EventBridge notifications on the S3 bucket, trigger a Step Functions state machine on Object Created events, and use a Choice state on $.detail.object.size. Files over 6 MB invoke ecs:runTask on Fargate; smaller files invoke Lambda.

Enable EventBridge notifications on the bucket with put-bucket-notification-configuration, then create an EventBridge rule matching source aws.s3 and detail-type Object Created. Add the state machine as a target with an IAM role allowing states:StartExecution.

Use Fargate when files exceed Lambda's limits, such as the 6 MB threshold used here or workloads needing more memory and longer runtimes. Lambda remains the better choice for small files due to faster startup and lower cost.

Verify EventBridge notifications are enabled on the bucket with get-bucket-notification-configuration, confirm the rule exists with aws events describe-rule, and check the target has the correct state machine ARN and an IAM role with states:StartExecution permission.

Fargate tasks stall when subnets lack auto-assign public IP or the route table has no 0.0.0.0/0 route to an internet gateway. Verify the security group allows outbound HTTPS and DNS so the task can pull the ECR image and write logs.

It is not suited for real-time streaming, where Kinesis is a better fit, and adds unnecessary orchestration if all files are small enough for direct S3-to-Lambda triggers. It also requires Docker to build and push the Fargate container image.