bullmq

Reliable Redis background jobs require configuring BullMQ queues and workers with defined attempts, backoff strategies, and removeOnFail settings to route terminal failures to a dead-letter queue. This ensures jobs don't get stuck or silently fail during execution.

BullMQ 5.x job scheduling requires a Redis client configuration with maxRetriesPerRequest set to null. This prevents the Redis client from failing commands during long polling operations, ensuring stable queue and worker connections for background processing.

Parent-child job pipelines for webhook fan-out are built using BullMQ's FlowProducer to orchestrate dependent job sequences. This allows parent jobs to spawn child jobs with individual retry policies and backoff, ensuring reliable multi-step asynchronous processing.

You can schedule repeating cron jobs and apply rate limiting in BullMQ by using queue.upsertJobScheduler for time-based triggers and configuring worker concurrency limits. This controls job throughput and prevents Redis overload during peak processing.

Background jobs stall when worker processes crash or block the event loop during CPU-intensive tasks. Handle this safely by enabling sandboxed CPU processing with useWorkerThreads or child processors, and implement stalled-job mitigation and graceful worker shutdown.

BullMQ supports job idempotency by assigning a custom jobId to prevent duplicate queue entries, and ensures deterministic cleanup by configuring removeOnComplete and removeOnFail thresholds. This protects Redis from out-of-memory risks during high-volume processing.