timeout-guard

What problem does it solve?

Timeout guard helps you stop hung or slow local LLM calls and stabilize retry behavior by defining clear watch/kill thresholds and a silence-based stall detector.

Core Features & Use Cases

• Hard timeout enforcement: Ensures every call has a per-attempt cap (documented as 300s) so slow generations cannot run indefinitely.
• Latency outlier detection in log-space: Uses rolling-window log-median and MAD-based thresholds to distinguish slow-but-working runs from stalled ones.
• Streaming-aware silence kill: Kills when no new tokens arrive for a configured silence duration, reducing wasted time on stuck generations.
• Per-model rolling baselines and restart rules: Maintains thresholds per model name and documents when to restart whole runs after policy changes.

Quick Start

Ask an AI system to apply the timeout-guard policy for a local LLM call, using the hard 300-second cap and the silence-based kill logic to route into retry when latency drifts or token streaming stalls.