using-redis-token-buckets

Implement atomic Redis token bucket rate limits with refund and Retry-After semantics.

713|118|Updated Aug 11, 2020
One-click install
npx skills add https://github.com/PostHog/posthog-foss --skill using-redis-token-buckets
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: using-redis-token-buckets
Source: https://github.com/PostHog/posthog-foss/tree/main/.agents/skills/using-redis-token-buckets
Command: npx skills add https://github.com/PostHog/posthog-foss --skill using-redis-token-buckets

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Fixed-window rate limiters suffer boundary bursts and full-window lockouts, and DRF throttles cannot refund charges for requests that did no work. This Skill guides you to implement a Redis-backed token bucket with continuous refill, atomic Lua operations, and accurate per-caller Retry-After values.

Core Features & Use Cases

  • Atomic consume/refund/peek API: Charge tokens, return them when a request did no work, and introspect budgets without charging, all via server-side Lua scripts over posthog.redis.get_client().
  • Accurate throttling semantics: Get real per-caller Retry-After waits and RateLimit-Limit/Remaining/Reset headers instead of window-edge guesses.
  • Use Case: You are adding a per-partner rate limit to an API endpoint where callers burst legitimately, failed validations must refund their token, and 429 responses need a truthful Retry-After header.

Quick Start

Add a Redis token bucket rate limit to my endpoint using posthog/token_bucket.py with a burst of 30 and 120 requests per hour, refunding the token when the request does no work.

Frequently Asked Questions about using-redis-token-buckets

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a Redis token bucket rate limit in PostHog?

Import consume, refund, and peek from posthog/token_bucket.py, define a Budget with burst and per_hour values, then call consume with a prefixed key like myfeature_rate:team_id. Handle BucketDecision and BucketUnavailable cases to throttle or fail open.

Token bucket vs fixed window rate limiting, which should I use?

Use a token bucket when callers burst legitimately, you need refunds for no-work requests, or Retry-After must reflect the real wait for the next token. Fixed-window counters suffer 2x boundary bursts and full-window lockouts.

When should I use DRF throttles instead of the token bucket?

Use the existing DRF throttles in posthog/rate_limit.py (IPThrottle, UserRateThrottle, PersonalApiKeyRateThrottle) for ordinary per-IP or per-user request throttling. They integrate with DRF's lifecycle and the RATE_LIMIT_ENABLED setting.

Does the token bucket survive a Redis flush or failover?

No, the bucket is best-effort; eviction or failover hands the caller a fresh budget. For hard quotas, pair it with a durable Postgres count and treat the bucket as the fast path.

How do I test Redis token bucket rate limits?

Under settings.TEST, posthog.redis.get_client() returns fakeredis which executes the Lua scripts. Control time with freeze_time, reset between tests with TEST_clear_clients() and TEST_reset_scripts(), and follow posthog/test/test_token_bucket.py.