forge-llm-streaming

What problem does it solve?

This Skill prevents flicker, broken Unicode, slow time-to-first-token, and non-working stop buttons when streaming LLM output to a UI, by enforcing a correct producer/consumer streaming model with proper transport, decoding, cancellation, and mid-stream error handling.

Core Features & Use Cases

• Transport-correct SSE streaming: Uses Server-Sent Events with the required headers to avoid proxy buffering and ensures each event is parseable on arrival.
• Structured event protocol: Streams typed events (delta/tool calls/done/error) with sequence numbers so the client can reassemble state reliably.
• Production-grade cancellation chain: Wires AbortController end-to-end (browser → server → upstream) so “stop” actually stops the LLM and prevents wasted tokens.
• Unicode-safe incremental decoding: Uses streaming TextDecoder behavior to avoid corrupt characters when multi-byte tokens cross chunk boundaries.
• Append-only UI rendering with backpressure discipline: Throttles re-renders, avoids full tree re-render per token, prevents auto-scroll yanks, and renders incrementally for responsiveness.
• Mid-stream errors as stream events: Returns errors after the HTTP 200 as in-band events, then renders them inline rather than relying on HTTP status codes.

Quick Start

Ask your AI coding agent to implement an SSE-based streaming endpoint and a browser reader that decodes UTF-8 incrementally, appends deltas, renders at paragraph boundaries, and propagates cancellation via AbortController.