async-task-platform-specialist

Build a distributed delay queue using Redis Sorted Sets for delay storage and Lua scripts for atomic job fetching. This ensures reliable scheduling by moving jobs through PENDING, ACTIVE, and COMPLETED states without race conditions.

Atomic fetch in a distributed delay queue uses a Lua script to simultaneously fetch and transition jobs into the ACTIVE state. This prevents multiple workers from pulling the same scheduled task from Redis concurrently.

Handle retries in a Redis delay queue using exponential backoff for rescheduling failed tasks. Jobs that exhaust retries are routed to a Dead Letter Queue, ensuring persistent failures are isolated without blocking active workers.

Recover stuck tasks in a distributed delay queue by implementing a watchdog system. The watchdog monitors ACTIVE jobs that exceed their expected processing time and automatically resets their state for re-processing.

Yes, you can scale worker pools in a Redis-based distributed delay queue. The architecture supports adding or removing workers dynamically because the atomic fetch mechanism coordinates job distribution centrally through Redis.

Production observability for a distributed delay queue requires tracking job lifecycle transitions, monitoring worker pool throughput, and alerting on DLQ accumulation. This ensures reliable operation and identifies scaling bottlenecks in the Redis backend.