quota-rate-limit-pattern

FastAPI gateway rate limiting can be implemented using a three-layer strategy: slowapi for burst RPM limits, asyncio Semaphore for concurrent in-flight admission, and DB-based daily token accumulation, returning distinct 429 reason codes for each layer.

OpenAI-compatible 429 errors require a structured error payload with a specific code field. You return distinct codes like rpm_limit, concurrent_limit, or daily_token_limit so clients can identify the exact quota layer blocking their request and retry appropriately.

Yes, asyncio Semaphore limits concurrent in-flight requests in ASGI gateways by controlling admission slots per user. This prevents backend overload from simultaneous connections, operating distinctly from slowapi RPM burst rate enforcement and daily token quota checks.

Tracking API rate limit rejections in Prometheus involves integrating a counter that labels rejections by reason code, such as rpm_limit, concurrent_limit, or daily_token_limit, enabling fast debugging of which quota layer is actively blocking traffic.

Worker-local rate limiting does not scale reliably for distributed FastAPI deployments because enforcement is isolated per worker. Moving beyond a single-worker setup requires Redis or an admission-controller alternative to coordinate quota and concurrency limits globally.

Separate mechanisms for burst rate, concurrency, and daily token quotas handle different failure modes: slowapi RPM manages sudden bursts, asyncio Semaphore controls active in-flight requests, and DB accumulation tracks long-term daily token overages reliably.