model-serving-security

Prevent token-exhaustion DoS by enforcing server-side parameter bounding on max_tokens, n, and logprobs. This caps request costs and stops a single unthrottled client from exhausting GPU capacity for all users.

SSRF protection validates and restricts URLs derived from model outputs before any outbound requests occur. It disables URL following by default unless explicitly enabled, preventing unintended server-side requests.

Implement multi-layer rate limiting by applying per-key, per-user, per-IP, and per-organization limits with shared state. This deters abuse and prevents multi-dimensional rate-limiting bypass across your endpoints.

Yes, you need streaming limits to secure model-serving endpoints. Streaming responses introduce timing and partial-response leakage channels, so enforcing server-side bounds on streaming prevents resource exhaustion and data leakage.

Yes, you can maintain availability during inference abuse by using real-time monitoring of latency, token usage, and error rates. Circuit-breaker behavior triggers automatically to protect endpoint availability under heavy load.

Unintended outbound requests occur when model-generated outputs include URLs that the serving layer follows. Enable SSRF-safe URL validation to restrict outbound requests before execution and disable URL following by default.