effect-uai-model-retry

Retry transient model failures such as RateLimited, Unavailable, and Timeout by applying an exponential backoff schedule with jitter to your model calls. This automatically handles temporary transport hiccups while capping the total retry attempts.

Exponential backoff with jitter is a retry strategy that progressively increases wait times between failed model calls while adding randomized jitter. This prevents thundering herd problems and manages rate limits safely across providers without overwhelming the endpoint.

Non-retryable errors like ContentFiltered, AuthFailed, InvalidRequest, and ContextLengthExceeded propagate immediately without triggering retries. A tagged error flow distinguishes these permanent failures from transient ones, ensuring only temporary issues trigger the backoff schedule.

This retry policy works with both single-turn and multi-turn model calls across various provider responses. It integrates as an inline pipeline without requiring a helper service, making it suitable for streaming and complex conversational workflows.

Use exponential backoff when experiencing transient failures like rate limits, transport hiccups, or timeouts. If errors are permanent such as InvalidRequest or AuthFailed, the system fails fast and surfaces the error immediately rather than retrying.

An inline retry pipeline operates without a helper service, meaning retry logic executes directly within the call flow. It caps retry limits safely but requires the application to manage the backoff schedule state directly during multi-turn or streaming workflows.