model-dos

What problem does it solve?

Protects LLM endpoints from resource exhaustion by enforcing input caps, per-user throttling, and strict timeouts.

Core Features & Use Cases

• Input length caps and token limits to bound prompts before they reach the model.
• Per-identifier throttling to prevent abuse from a single user or API key.
• Explicit inference deadlines to ensure timely responses even when upstreams misbehave.
• Applicable to any service exposing an LLM API, including chatbots and multi-turn workflows.

Quick Start

Configure your LLM endpoint to enforce max tokens, rate limits, and request timeouts for incoming prompts.