async-dedup-race-in-streaming-pipelines

What problem does it solve?

Streaming pipelines often emit multiple results for the same dedup key when per-item async enrichment is performed after a pre-check, causing invisible or duplicate entries in UI and incorrect counts.

Core Features & Use Cases

• Synchronous dedup reservation: reserve the slot before spawning async work to prevent duplicates.
• Await all async tasks before done: avoid premature completion by waiting on Promise.all for all in-flight work.
• Cache shared resources: reuse I/O results across concurrent requests to reduce redundant work.
• Use cases include SSE/WebSocket streaming dashboards, real-time analytics pipelines, and UI lists that require strict one-emission-per-key guarantees.

Quick Start

Run the async dedup race fixes on your streaming pipeline to ensure that each dedup key yields at most one result and that all async processing completes before signaling pipeline completion.