processing-s3-uploads-with-step-functions

Enable EventBridge notifications on the S3 bucket, trigger a Step Functions state machine on Object Created events, and use a Choice state on $.detail.object.size. Files over 6 MB run an ECS Fargate task; smaller files invoke Lambda.

Enable EventBridge notifications on the bucket with put-bucket-notification-configuration, then create an EventBridge rule matching source aws.s3 and detail-type Object Created. Add the state machine as a target with an IAM role allowing states:StartExecution.

You need AWS CLI v2 configured with valid credentials, Python 3.12 for the Lambda runtime, and Docker to build and push the Fargate container image to ECR. A default VPC with two subnets is also required.

Verify EventBridge notifications are enabled on the bucket with get-bucket-notification-configuration, confirm the rule exists with aws events describe-rule, and check the target has the correct state machine ARN and IAM role.

Skip this approach when all files fit Lambda's limits, since a direct S3 to Lambda trigger is simpler. It is also unsuitable for real-time streaming workloads, where Kinesis is the appropriate service.

Fargate tasks stall when subnets lack auto-assign public IP or the route table has no internet gateway route for 0.0.0.0/0. Verify the security group allows outbound HTTPS and DNS traffic for image pulls and logging.