API Rate Limiting: Token Buckets, Sliding Windows, and Fair Usage
Rate limiting done well is an algorithm (token bucket vs sliding window vs fixed window) plus a contract: 429 with Retry-After. A token bucket implemented and tested from scratch, then Laravel's throttle hit with real requests until it pushed back.
Table of contents
Every public API eventually meets the client that sends ten thousand requests a minute - a buggy retry loop, an over-eager crawler, a competitor scraping your catalog. Rate limiting is how you decide, in advance, what happens next: the abusive client gets a clear "slow down" signal instead of consuming resources the rest of your traffic needs. Done well it's two things at once: an algorithm that decides who is over the line, and a contract (HTTP 429 + Retry-After) that tells clients how to behave. Below: the three algorithms worth knowing, a token bucket implemented and tested from scratch, and Laravel's limiter hit with real requests until it pushes back.
The three algorithms, honestly compared#
Fixed window counter. Count requests per key per window. In the classic clock-aligned variant the window is the wall-clock minute ("14:03"): simple, tiny state - and a known edge case, where a client fires a full allowance at 14:03:59 and another at 14:04:00, getting 2x the intended rate across the boundary. (Framework implementations may instead start the window at the first hit; the boundary burst is specific to the clock-aligned variant.) If a brief burst can't hurt you, that may be an acceptable price for the simplicity.
Sliding window. Look back over the last N seconds instead of aligning to the clock. The precise version stores a timestamp per request (a log); the common approximation weights the previous window's counter by how much of it is still in the look-back period. Smoother than fixed windows, at the cost of more state or more math.
Token bucket. A bucket holds up to capacity tokens and refills at a steady rate; each request spends tokens. The two knobs map directly to product language: capacity is the burst you're willing to tolerate, refill rate is the sustained rate you intend to serve long-term. A client who's been quiet gets a quick burst; a client who keeps hammering drains the bucket and falls back to the refill rate. It's the usual default when you want "be nice most of the time, sprint occasionally."
(A leaky bucket is the mirror image: requests are metered out at a fixed rate. The textbook version queues requests and drains the queue; variants reject instead of queueing. Either way the point is smoothing what reaches a fragile downstream.)
| Algorithm | Burst behavior | State per key | Boundary problem |
|---|---|---|---|
| Fixed window | Full allowance at any moment in the window | One counter | Yes - 2x across the edge |
| Sliding window (approx.) | Smoothed | One or two counters | Mostly smoothed |
| Sliding window (log) | Exact | One timestamp per request | No |
| Token bucket | Up to capacity, then refill rate | Tokens + last-refill time | No |
A token bucket you can test#
The algorithm is small enough to implement exactly - and small enough to cover with focused tests. Two design choices do the heavy lifting: the clock is injected (so tests are deterministic), and backwards time is rejected loudly instead of silently producing negative elapsed time:
declare(strict_types=1);
namespace App\Support;
/**
* A token bucket: `capacity` tokens max, refilled at `refillPerSecond`.
*
* The clock is injected so tests are deterministic. In production pass a
* MONOTONIC clock (hrtime(true) / 1e9), not a wall clock like
* microtime(true): NTP adjustments can move wall time backwards, which
* this class rejects rather than silently absorbing.
*/
final class TokenBucket
{
private function __construct(
private readonly int $capacity,
private readonly float $refillPerSecond,
private float $tokens,
private float $lastRefillAt,
) {
if ($capacity < 1) {
throw new \DomainException('Capacity must be a positive integer');
}
if (! is_finite($refillPerSecond) || $refillPerSecond <= 0) {
throw new \DomainException('Refill rate must be a positive finite number');
}
self::assertFiniteTime($lastRefillAt);
}
public static function full(int $capacity, float $refillPerSecond, float $now): self
{
return new self($capacity, $refillPerSecond, $capacity, $now);
}
/** Take $cost tokens if available; refill first based on elapsed time. */
public function tryConsume(float $now, int $cost = 1): bool
{
$this->assertCost($cost);
$this->refill($now);
if ($this->tokens < $cost) {
return false;
}
$this->tokens -= $cost;
return true;
}
/** Seconds until $cost tokens are available, or null if available now. */
public function retryAfterSeconds(float $now, int $cost = 1): ?float
{
$this->assertCost($cost); // a cost above capacity can NEVER be served: no finite wait
$this->refill($now);
if ($this->tokens >= $cost) {
return null;
}
return ($cost - $this->tokens) / $this->refillPerSecond;
}
public function tokens(float $now): float
{
$this->refill($now);
return $this->tokens;
}
private function assertCost(int $cost): void
{
if ($cost < 1 || $cost > $this->capacity) {
throw new \DomainException("Cost must be between 1 and the bucket capacity ({$this->capacity})");
}
}
private static function assertFiniteTime(float $t): void
{
if (! is_finite($t)) {
throw new \DomainException('Time must be a finite number');
}
}
private function refill(float $now): void
{
self::assertFiniteTime($now);
if ($now < $this->lastRefillAt) {
throw new \DomainException('Clock moved backwards'); // never rewind time silently
}
$elapsed = $now - $this->lastRefillAt;
$this->tokens = min($this->capacity, $this->tokens + $elapsed * $this->refillPerSecond);
$this->lastRefillAt = $now;
}
}Two details worth noticing: both public methods validate the cost against the capacity - a request for more tokens than the bucket can ever hold gets a loud DomainException, not a polite but impossible "wait 40 seconds" - and the clock must be monotonic (hrtime(true) / 1e9 in production), because a wall clock can jump backwards under NTP adjustments and this class refuses to absorb that silently. The 8 tests cover the burst-then-reject contract, refill math, the capacity cap, the exact retry-after wait, and every invalid input named above (zero, negative, and over-capacity costs exercised against both public methods):
declare(strict_types=1);
namespace Tests\Unit;
use App\Support\TokenBucket;
use PHPUnit\Framework\TestCase;
class TokenBucketTest extends TestCase
{
public function test_full_bucket_allows_burst_up_to_capacity_then_rejects(): void
{
$bucket = TokenBucket::full(capacity: 5, refillPerSecond: 1.0, now: 1000.0);
for ($i = 0; $i < 5; $i++) {
$this->assertTrue($bucket->tryConsume(1000.0));
}
$this->assertFalse($bucket->tryConsume(1000.0));
}
public function test_tokens_refill_with_elapsed_time(): void
{
$bucket = TokenBucket::full(5, 2.0, 1000.0);
$bucket->tryConsume(1000.0, 5); // empty it
$this->assertFalse($bucket->tryConsume(1000.4)); // only 0.8 tokens refilled
$this->assertTrue($bucket->tryConsume(1001.0)); // 2.0 available now
$this->assertEqualsWithDelta(1.0, $bucket->tokens(1001.0), 1e-9);
}
public function test_refill_never_exceeds_capacity(): void
{
$bucket = TokenBucket::full(5, 10.0, 1000.0);
$this->assertSame(5.0, $bucket->tokens(1999.0)); // ages later: still capped at 5
}
public function test_retry_after_is_the_exact_wait_for_the_next_token(): void
{
$bucket = TokenBucket::full(5, 2.0, 1000.0);
$bucket->tryConsume(1000.0, 5);
$this->assertSame(0.5, $bucket->retryAfterSeconds(1000.0)); // need 1 token at 2/s
$this->assertNull($bucket->retryAfterSeconds(1000.5)); // token available
}
public function test_invalid_costs_are_rejected_by_both_methods(): void
{
$bucket = TokenBucket::full(5, 1.0, 1000.0);
// 6 is over capacity: it could never be served, so no finite wait exists.
foreach ([0, -1, 6] as $cost) {
foreach (['tryConsume', 'retryAfterSeconds'] as $method) {
try {
$bucket->$method(1000.0, $cost);
$this->fail("$method accepted cost $cost");
} catch (\DomainException) {
$this->addToAssertionCount(1);
}
}
}
}
public function test_backwards_clock_is_rejected(): void
{
$bucket = TokenBucket::full(5, 1.0, 1000.0);
$this->expectException(\DomainException::class);
$bucket->tryConsume(999.0);
}
public function test_invalid_construction_is_rejected(): void
{
foreach ([0, -3] as $capacity) {
try {
TokenBucket::full($capacity, 1.0, 1000.0);
$this->fail("capacity $capacity was accepted");
} catch (\DomainException) {
$this->addToAssertionCount(1);
}
}
foreach ([0.0, -1.0, NAN, INF] as $rate) {
try {
TokenBucket::full(5, $rate, 1000.0);
$this->fail('refill rate '.var_export($rate, true).' was accepted');
} catch (\DomainException) {
$this->addToAssertionCount(1);
}
}
}
public function test_non_finite_time_is_rejected(): void
{
$bucket = TokenBucket::full(5, 1.0, 1000.0);
$this->expectException(\DomainException::class);
$bucket->tryConsume(INF);
}
}
Time: 00:00.002, Memory: 20.00 MB
OK (8 tests, 26 assertions)Laravel's limiter, hit with real requests#
Laravel ships a cache-backed rate limiter and a throttle middleware. The smallest honest demo: a route limited to 5 requests per minute per client, hit 7 times, then inspected:
// Rate-limit demo route: 5 requests per minute per IP.
Route::get('/rate-demo', function () {
return response()->json(['ok' => true, 'time' => now()->toTimeString()]);
})->middleware('throttle:5,1');Real run against the app (PHP 8.4.27, Laravel 13, requests sent sequentially - no concurrency is simulated or claimed). The second command asks for JSON, which is what your API clients will actually send:
$ for i in 1 2 3 4 5 6 7; do curl -s -o /dev/null -w "request $i: %{http_code}\n" http://127.0.0.1:8799/rate-demo; done
request 1: 200
request 2: 200
request 3: 200
request 4: 200
request 5: 200
request 6: 429
request 7: 429
$ curl -si -H "Accept: application/json" http://127.0.0.1:8799/rate-demo
(transcript trimmed: cookie and other unrelated response headers omitted)
HTTP/1.1 429 Too Many Requests
X-RateLimit-Limit: 5
X-RateLimit-Remaining: 0
Retry-After: 59
X-RateLimit-Reset: 1794245626
Content-Type: application/json
{
}The first five requests pass; the sixth and seventh get 429 Too Many Requests. The JSON body here is an empty {} because this run used APP_DEBUG=false, matching production behavior (a debug run adds message and even a stack trace - which is one more reason debug stays off in production). The headers carry the actual contract your clients will code against:
x-ratelimit-limit: 5/x-ratelimit-remaining: 0- the allowance and where this key stands in it.retry-after: 59- seconds until this key may retry; it counts down as the window progresses. A well-behaved client waits at least this long instead of guessing.x-ratelimit-reset- the Unix timestamp when the window resets.
throttle:5,1 is the terse form; for anything real, define a named limiter in RateLimiter::for('api', ...) and choose the key deliberately - authenticated user id, API key, or IP - because the key is the fairness policy. In this anonymous demo the key fell back to the request's IP as the app saw it; behind a load balancer, your trusted-proxy configuration decides whether that IP is the client's or the balancer's, so verify it before trusting per-IP limits.
One more honesty note: counters in a shared cache store mean shared storage, not automatically an atomic decision. A check-then-increment done as two operations has a race window, and what your framework and driver actually guarantee depends on their implementation. This demo ran sequentially and proves the request/response behavior only - if you need a strict cap under concurrency, verify that your limiter performs admission as one atomic operation covering the counter and its window/expiry handling (how that looks depends on the store and driver you run), and test it under real concurrency.
Fair usage is a product decision, not just math#
- Pick the key on purpose. Per-IP limits are the common fallback for anonymous traffic but punish everyone behind one NAT or office gateway; per-user or per-API-key limits are fairer but do nothing before authentication. Many APIs do both: a tight anonymous per-IP limit, a roomier authenticated one.
- Weight expensive requests. A search that scans an index and a health check shouldn't cost the same. Token buckets make this natural: the expensive endpoint just costs more tokens.
- Tiers beat one-size-fits-all. Free, pro, and internal clients are different constituencies with different budgets; named limiters per tier keep the policy legible.
- Make 429 usable. RFC 6585 defines 429 and makes
Retry-Afteroptional - for your own API, make it non-optional in your contract, keep the error body consistent and machine-readable, and document the limits where developers will actually read them. The correct client behavior is: wait at least theRetry-Aftervalue, then add jittered backoff if 429s continue. Retrying sooner almost certainly earns another 429 - the limiter rejects, it doesn't queue. - A limiter complements your security controls. It raises the cost of abuse and absorbs accidents; it does not stop a determined attacker with many keys or IPs, and it works alongside - not instead of - authentication, authorization, and input validation.
Common mistakes#
| Mistake | What it costs |
|---|---|
| Fixed windows where bursts matter | 2x your "limit" delivered at every window edge |
| In-memory counters behind a load balancer | Each server enforces its own limit; effective limit = limit x servers |
| Non-atomic check-then-increment | Concurrent requests both pass; your limit is a suggestion under load |
429 without Retry-After |
Clients guess, retry immediately, and make the overload worse |
| Keying everything by IP | One shared office or carrier NAT gets throttled as one "user" |
| Limiting the health check into uselessness | Monitoring flaps during incidents, when you need it most |
| Silent limiting (dropping/delaying without a 429) | Clients can't tell throttling from a broken API |
Checklist#
- Algorithm chosen for your burst tolerance (token bucket if bursts are fine, sliding window if not)
- Counter storage shared across all app servers - and check-and-decrement verified atomic under your framework/driver, not assumed
- Key per authenticated identity where possible, per-IP as the anonymous fallback (with trusted proxies configured)
- Your API's 429 responses always carry
Retry-After(optional in the RFC, non-optional in your contract) and a consistent body - Limits documented; clients told to back off with jitter
- Expensive endpoints cost more than cheap ones
- Monitoring and health checks exempted or generously budgeted
- Limit behavior tested with real requests, not just unit math
Key takeaways#
Rate limiting is an algorithm plus a contract. Token buckets give you a tunable burst allowance on top of a bounded sustained rate; sliding windows give you smoothness at the cost of state; fixed windows give you simplicity with a known boundary burst in the clock-aligned variant. Whatever you pick, the limit only holds if the counter is shared across your fleet and the check-and-decrement is atomic - shared storage alone doesn't decide anything - and the feature only works if clients can understand the answer: 429, Retry-After, and documentation. Test the math in isolation, then test the middleware with real requests until it pushes back.
References#
- Laravel rate limiting: https://laravel.com/docs/rate-limiting
- HTTP 429 and Retry-After semantics: https://www.rfc-editor.org/rfc/rfc6585#section-4
- Why your indexes matter for what sits behind the API: /en/blog/why-your-query-is-slow-mysql-indexes
- Zero-downtime schema changes for the tables your API serves: /en/blog/zero-downtime-mysql-migrations
Designing an API that has to survive real traffic? See /en/services.