How Accurate Should a Distributed Rate Limiter Be When the Limiter Itself Can Fail?

A rate limiter sounds like arithmetic.

Allow:

100 requests / minute

Count requests, and when request 101 arrives:

HTTP 429 Too Many Requests

That model works beautifully while the application runs on one machine and one process owns the counter.

Then the service scales horizontally.

Now the obvious question appears:

Where does the counter live?

If every instance maintains its own state, a client with a nominal limit of:

100 requests/minute

may effectively receive:

100 × number_of_instances

requests before the fleet agrees that the limit has been exceeded.

So the counter moves into Redis.

That fixes one problem.

Then Redis gets slow. Or a replica is stale. Or a shard fails. Or one tenant becomes such a hot key that the rate limiter itself becomes the bottleneck. Or a network partition prevents part of the API fleet from reaching the state used to decide whether requests should be admitted.

At that point, the interesting question is no longer:

“Which rate-limiting algorithm should we use?”

It becomes:

“How much enforcement accuracy are we willing to sacrifice so the mechanism protecting availability does not itself destroy availability?”

That is what makes distributed rate limiting interesting.

The implementation may contain an INCR.

The architecture contains a consistency and failure policy.

The Counter Becomes a Consistency Problem

Start with one Node.js process and a simple per-tenant rule:

100 requests / second

A local implementation might look like:

const counters = new Map<string, { count: number; resetAt: number }>();

function allow(tenantId: string, now = Date.now()): boolean {
  const current = counters.get(tenantId);

  if (!current || now >= current.resetAt) {
    counters.set(tenantId, {
      count: 1,
      resetAt: now + 1_000,
    });

    return true;
  }

  if (current.count >= 100) {
    return false;
  }

  current.count++;

  return true;
}

For one process, the model is straightforward:

client
→ one counter
→ one decision

The local state is authoritative.

Now deploy several instances behind a load balancer.

Each server independently sees:

less than 100

and allows traffic.

Globally:

70 + 62 + 68 = 200

requests have already occurred.

The limiter has become approximate without anyone intentionally deciding that approximation was acceptable.

That is the first distributed-rate-limiting problem:

The thing being limited is global, while the observations are local.

Shared coordination can fix that.

Shared coordination also introduces latency, availability, consistency, and failure-domain costs.

Those costs become part of the limiter design just as much as the quota itself.

Redis Solves Coordination, Not the Entire Design

A shared Redis deployment gives the API fleet one logical counter:

Each request can perform something like:

INCR ratelimit:tenant_42

Now the instances agree on one count.

Redis is attractive here because increment operations are atomic, and its documentation explicitly presents rate limiting as a common use case. When the limiter requires several operations to make one decision, server-side atomic execution through Lua or an equivalent primitive can prevent clients from racing through a read/decide/update sequence.

A fixed-window key might look like:

ratelimit:tenant_42:202610022343

with an expiration.

A simple implementation could be:

const count = await redis.incr(key);

if (count === 1) {
  await redis.expire(key, 60);
}

return count <= limit;

The code is small.

The edge cases are not.

Two commands can create one race

Suppose:

INCR key

succeeds and the application crashes before:

EXPIRE key

executes.

The counter may now live far longer than intended.

Redis documents this kind of race, which is why the increment and first expiration are commonly made atomic.

For example:

local current = redis.call("INCR", KEYS[1])

if current == 1 then
    redis.call("EXPIRE", KEYS[1], ARGV[1])
end

return current

Now:

increment
+
initial expiration

form one atomic server-side operation from the clients' perspective.

Current Redis versions also provide newer primitives such as INCREX that can combine increment, bounds, and expiration behavior in a single command.

That solves an important local atomicity problem.

It still does not answer what happens when Redis is stale, unavailable, partitioned, or overloaded.

Choose the Algorithm Around the Failure You Care About

Algorithm discussions often begin with:

fixed window
sliding window
token bucket
leaky bucket

I think that conversation is more useful after we define what the limiter is trying to protect.

Fixed windows are simple but burst around boundaries

Suppose the policy is:

100 requests / minute

A client sends 100 requests at:

12:00:59

and another 100 at:

12:01:00

Both windows are valid.

The client has effectively sent:

200 requests

across roughly one second.

timeline
    title Fixed Window Boundary
    12:00:59 : 100 requests allowed
    12:01:00 : Counter resets
             : 100 more requests allowed

That may be completely acceptable for:

billing-plan fairness

It may be dangerous if the limiter exists because:

the database cannot safely absorb
200 requests in one burst

The algorithm should follow the actual failure mode.

Token buckets separate sustained rate from burst capacity

A token bucket lets us express:

sustained rate = 100 req/s
burst capacity = 150 requests

Tokens regenerate over time and requests consume them:

Stripe has described using token-bucket rate limiting in production, including temporary burst capacity above a steady rate.

That distinction is useful because real traffic is rarely perfectly smooth.

A tenant doing:

0
0
0
40 requests
0

may still be perfectly well-behaved even if its long-term target rate is modest.

Rejecting every short burst can reduce usability without meaningfully protecting the system.

The important question is therefore not whether one algorithm is theoretically superior.

It is whether the chosen algorithm represents the resource or policy the limiter is actually trying to enforce.

Rate, Concurrency, and Load Are Different Signals

Request rate is only one kind of pressure.

Suppose one endpoint takes:

10 ms

and another takes:

10 seconds

Both receive:

10 requests/sec

The cheap endpoint has roughly:

0.1 requests

concurrently active on average.

The expensive endpoint can have around:

100 requests

active at once.

The request rate is identical.

The resource pressure is completely different.

Stripe has described using separate concurrent-request limiters for expensive operations because requests-per-second limits alone did not adequately control resource contention.

That gives us an important distinction:

rate limit
=
how quickly new work may enter

versus:

concurrency limit
=
how much work may be in progress

Latency can invalidate an apparently safe RPS limit

Suppose an operation normally takes:

5 seconds

and the service can safely execute:

20 concurrently

A rate cap of:

4 requests/sec

appears consistent with that capacity.

Then dependency latency doubles:

10 seconds

and:

4 req/sec × 10 sec
≈ 40 concurrent requests

The service overloads without violating the request-rate limit at all.

A concurrency limit adapts naturally:

20 active
→ request 21 is rejected or queued

If the protected resource fails because too much work is simultaneously active, concurrency is often the more direct safety signal.

Rate limiting is not load shedding

Now suppose every tenant remains below its quota, but database latency triples.

No customer is abusing the API.

The fleet can still be overloaded.

Static quotas answer:

How much traffic may this caller send?

Load shedding answers:

How much work can the system accept right now?

Those are different decisions.

Stripe describes using several layers together: request-rate limits, concurrent-request limits, and fleet-level load shedding that preserves capacity for more critical API operations.

A mature design should not assume:

rate limiter present
=
overload solved

Rate limiting is one form of admission control.

It is not the entire capacity-control strategy.

The Hard Question Is What Happens When the Limiter Fails

Now return to the shared-state architecture:

API instances
→ Redis
→ allow / deny

Every request depends on Redis.

Redis becomes unavailable.

What should happen?

There are two obvious extremes.

Fail closed

cannot check limiter
→ reject request

Fail open

cannot check limiter
→ allow request

Neither is universally correct.

Failing closed preserves enforcement

Consider a login endpoint where the limiter exists to reduce:

credential stuffing
brute-force attempts

If Redis disappears and the application responds by allowing unlimited attempts, the security property disappears exactly while one of the components enforcing it is unhealthy.

For that kind of boundary, failing closed—or falling back to another restrictive local mechanism—may be appropriate.

The cost is obvious:

Redis outage
→ login availability decreases

That may still be the correct trade.

What matters is that it is intentional.

Failing open preserves application availability

Now consider a soft fairness quota on an ordinary API.

If the limiter is unreachable for 20 seconds, rejecting every API call could create a much larger outage than temporarily allowing a few clients to exceed their normal quotas.

Stripe explicitly recommends integrating limiters so defects in limiter logic or Redis failure do not unnecessarily take down the API itself.

In that case:

limiter unavailable
→ allow request
→ emit degraded signal

may be a better policy.

The key principle is:

Fail-open versus fail-closed is a product and reliability decision, not a Redis decision.

Different endpoints can reasonably choose different answers.

A single global:

catch {
  return true;
}

or:

catch {
  return false;
}

usually throws away too much policy information.

Local Fallback Trades Precision for Availability

A useful middle ground is:

shared limiter healthy
→ use Redis-backed global policy

shared limiter unavailable
→ use a coarse local emergency policy

For example:

async function allowRequest(key: string): Promise<boolean> {
  try {
    return await distributedLimiter.allow(key);
  } catch {
    return localEmergencyLimiter.allow(key);
  }
}

Suppose ten application instances each fall back to:

20 requests/sec

The fleet might temporarily allow something near:

200 requests/sec

instead of the intended global limit.

That is inaccurate.

It may still be substantially safer than:

unlimited traffic

or:

zero traffic

The architecture has consciously traded precision for availability.

This is an important shift in mindset: during degraded operation, the goal may not be to preserve the exact nominal quota. The goal may be to preserve the reason the quota exists.

If the limiter exists for fairness, a temporary 10–20% overshoot may be tolerable.

If it exists to prevent one database from collapsing, the local fallback needs to remain conservative enough to protect that database even when global coordination is gone.

If it exists as a security boundary, approximation may be unacceptable.

The fallback should preserve the invariant, not merely imitate the normal algorithm.

Perfect Global Accuracy Requires Global Coordination

Now spread the API across several regions:

Virginia
Oregon
Frankfurt
Tokyo
Sydney

One tenant has a truly global quota:

1,000 requests/sec

To enforce that perfectly, every region needs sufficiently fresh knowledge of what all the others have admitted inside the same logical window.

That implies coordination.

The benefit is improved accuracy.

The costs may include:

higher request latency
cross-region dependency
larger blast radius
lower partition tolerance

If the shared limiter becomes unreachable, several otherwise independent regions may be affected together.

The rate limiter has become a global failure domain.

That might be justified if the quota represents a hard security or correctness invariant.

It may be an unnecessary price for an ordinary fairness policy.

Perfect enforcement is not free.

The more global the truth needs to be, the more coordination the system usually needs to buy.

Approximation Can Be the Better Engineering Choice

Cloudflare encountered the general problem while designing rate limiting across millions of domains.

Purely local counters were too inaccurate because requests could land on different machines.

Globally centralizing the counter for every request would have introduced undesirable coordination and latency.

Cloudflare instead localized counting within a point of presence and used an approximate sliding-window algorithm. In an analysis covering roughly 400 million requests from 270,000 sources, it reported around 0.003% incorrect limiting decisions for that workload.

That result illustrates a broader principle:

A slightly approximate limiter that is fast, local, and available may be better than a mathematically exact limiter that introduces a fragile distributed dependency into every request.

The useful question is not:

Is the limiter exact?

It is:

How often is it wrong?

In which direction?

What does that error cost?

False allows and false rejects have different consequences

An approximate limiter can make two kinds of mistakes.

A false allow means:

request should have been rejected
but was admitted

A false reject means:

request should have been admitted
but was rejected

For a soft fairness quota, a false allow may be nearly harmless while a false reject creates unnecessary customer pain.

For a brute-force defense, the cost may be reversed.

Cloudflare's analysis is useful partly because it measured incorrect decisions rather than treating approximation as an abstract property.

Approximation should be evaluated economically and operationally.

Not philosophically.

Ask what being wrong by 5% means

Suppose the intended limit is:

100 requests/sec

and the distributed implementation occasionally permits:

105 requests/sec

For:

free-tier fairness

that may not matter.

For:

a dependency that becomes unstable
above 100 concurrent operations

it may matter greatly.

This is why fairness quotas and hard capacity boundaries should not automatically share the same consistency design.

Accuracy is a product requirement.

Not every rate limit deserves the same one.

Regional Budgets Trade Utilization for Independence

One way to avoid globally coordinating every request is to divide a global budget into regional allocations.

For:

global quota = 1,000 req/sec

we might assign:

US      400
EU      250
Asia    250
Other   100

Each region can now make decisions locally.

That improves:

latency
regional independence
partition tolerance
blast-radius containment

But suppose traffic shifts:

US needs 700
Asia needs 20

Capacity can become stranded in a quiet region while another rejects useful traffic.

Regional budgets trade:

global utilization efficiency

for:

local independence

That may be a very good trade.

It is still a trade.

A more adaptive system can periodically rebalance budgets, but that reintroduces coordination—just at a slower control-plane timescale rather than on every request.

That is often a much better place to pay the coordination cost.

Replication and Sharding Introduce New Failure Modes

A Redis-backed limiter can be locally atomic and still globally inconsistent.

Suppose Redis uses:

primary
+
read replicas

Writes go to the primary.

Reads can come from replicas.

Replication is not necessarily instantaneous.

Request A updates the primary so that:

remaining = 0

Request B reads a lagging replica and still sees:

remaining = 2

and allows traffic that a perfectly current view would have rejected.

GitHub encountered this class of complexity while moving API rate limiting to sharded and replicated Redis. Its design used primaries for writes, replicas for reads, application-side sharding, and Lua for atomic operations, which then required dealing with stale state and inconsistencies visible in rate-limit metadata.

The lesson is simple:

Atomicity on one Redis node does not make the entire distributed limiter strongly consistent.

The decision and its metadata should share a snapshot

Clients often receive headers such as:

X-RateLimit-Limit
X-RateLimit-Remaining
X-RateLimit-Reset

Imagine the server:

1. evaluates the request,
2. rejects it,
3. performs another Redis lookup to generate headers.

If the rate-limit window resets between steps 1 and 3, the response might effectively say:

Request rejected because the limit was exceeded.

Remaining: 100

GitHub encountered this kind of inconsistency and changed its design so the limiting decision and the metadata explaining that decision came from the same operation.

That reveals a useful general rule:

If a decision and the explanation of that decision must agree, derive them from the same snapshot of state.

Hot tenants become hot Redis keys

Suppose most tenants generate:

10 requests/sec

while one very large tenant generates:

100,000 requests/sec

All of those decisions may touch:

rate:tenant:enterprise_42

Sharding distributes many keys across the cluster:

Cluster-wide CPU can look healthy while shard 2 is saturated.

This is the same lesson that appears in Cassandra, caching, queues, and many other distributed systems:

Balanced keys do not guarantee balanced work.

Sharding complicates multi-dimensional limits

Suppose one request must satisfy:

per-user limit
+
per-tenant limit
+
global load limit

Those counters may live on different Redis shards.

One Lua script cannot atomically manipulate arbitrary keys spread across independent shards.

You now have to decide whether to:

co-locate related counters
perform independent checks
accept partial updates
split admission into multiple stages

For example:

Each stage can have a different scope, consistency requirement, and failure policy.

Trying to turn every form of rate limiting into one globally atomic transaction usually creates a worse architecture.

Identity and Cost Determine What the Limit Means

A limiter only makes sense once we define who or what is consuming the budget.

Possible keys include:

IP address
user ID
API key
tenant ID
device ID
endpoint

Each represents a different policy.

IP-based limits can unfairly penalize:

large offices
universities
carrier NAT
shared proxies

because many legitimate users may share one public address.

User-level limits require authentication before the limiter can identify the user.

API-key limits fit developer APIs, but a compromised key can consume the account's entire budget.

Tenant-level limits protect platform fairness while still allowing one user inside the tenant to monopolize the shared allowance.

Many systems therefore need several layers:

IP abuse protection
+
user/account quota
+
tenant quota
+
endpoint-specific safety control

Rate limiting is policy expressed through counters.

One request is not necessarily one unit of cost

Consider:

GET /profile

CPU: 1 ms
DB: one indexed read

and:

POST /reports/export

CPU: 2 seconds
DB: millions of rows scanned

If both consume:

1 token

the limiter is pretending that they cost the system the same amount.

One alternative is weighted accounting:

/profile        = 1 token
/search         = 3 tokens
/report/export  = 50 tokens

with something like:

await limiter.consume({
  key: tenantId,
  cost: endpointCost,
});

That can work.

It also requires a credible model of cost.

For extremely expensive operations, a direct concurrency limit may be simpler and more closely coupled to the resource being protected.

The right unit is not always "one request."

A Rejection Is Part of a Feedback Protocol

Returning 429 is not the end of the design.

The client needs to know what to do next.

A response may include:

HTTP/1.1 429 Too Many Requests
Retry-After: 8

The client should not interpret that as:

429
→ retry immediately
→ 429
→ retry immediately
→ 429

because the rate limiter has now become the source of a retry storm.

Rate limiting works best as a feedback loop:

clear rejection
+
retry guidance
+
backoff
+
jitter

The exact retry policy depends on the API, but the principle is important:

Rate limiting is communication about capacity, not merely rejection.

Retry timing can synchronize the next spike

Suppose:

10,000 clients

all receive:

Retry-After: 10

and every client retries at exactly 10 seconds.

The system has moved one burst ten seconds into the future.

Adding jitter spreads that recovery traffic:

retry after approximately 10 seconds
+
random variation

Capacity-control mechanisms interact.

A sophisticated server-side limiter can still be defeated by a poorly behaved client retry strategy.

Observe the Limiter as Its Own Distributed System

A rate limiter that sits on every request is production infrastructure.

It deserves its own telemetry.

At minimum, I would want:

rate_limit_checks_total
rate_limit_allowed_total
rate_limit_rejected_total
rate_limit_backend_errors_total
rate_limit_fallback_total
rate_limit_latency

with useful dimensions such as:

limiter type
endpoint
tenant class
region
failure mode

A healthy snapshot might look like:

request limiter rejections:      2.1%
concurrency limiter rejections:  0.02%
Redis errors:                    0
local fallback decisions:        0

During a Redis incident:

Redis errors:                18%
local fallback decisions:    18%
API availability:          99.9%

Now operators can see that:

limiter degraded

without concluding:

API unavailable

That is far better than learning:

Redis is down

from:

the entire API is down

Measure approximation explicitly

If the architecture intentionally allows approximate decisions, I want to know how often those decisions differ from the ideal reference behavior.

Measure:

false allows
false rejects

separately.

Those errors can carry very different costs depending on the limiter's purpose.

The observability should match the policy.

A security limiter should surface enforcement gaps differently from a soft fairness quota.

Build the Policy Before Choosing the Algorithm

Before choosing:

fixed window
sliding window
token bucket

I would first answer:

What resource or policy is this protecting?

What happens if we allow too much?

What happens if we reject too much?

How global does the limit really need to be?

How much burst is acceptable?

What happens if the state store fails?

Can enforcement be approximate?

Should failure be open, closed,
or locally degraded?

Do we need to control rate,
concurrency, or current fleet load?

Only then would I choose the algorithm.

Otherwise, the design conversation starts one abstraction too low.

A token bucket is an implementation of a policy.

It is not the policy itself.

Accuracy is a product requirement

Consider three limits.

A free-tier API quota:

1,000 requests/day

If races occasionally permit:

1,004

that may be irrelevant.

A database-protection limit:

maximum 100 concurrent exports

If the system occasionally allows:

300

the database may become unstable.

A login-abuse control:

5 failed attempts/minute

If limiter failure suddenly allows:

100

the security assumption has disappeared.

All three are described as:

rate limits

but they have radically different correctness requirements.

That is why I would define the cost of enforcement error before deciding how strongly consistent the counter needs to be.

A Practical Layered Design

For a multi-tenant API, I would think about admission control as several independent stages:

Not every request needs every stage.

Each stage answers a different question.

IP limiter:
Is this source obviously abusive?

User limiter:
Is this principal behaving reasonably?

Tenant limiter:
Is one customer monopolizing shared capacity?

Concurrency limiter:
Can this expensive operation start safely?

Load shedder:
Can the fleet accept this work right now?

That is much more expressive than one global:

100 req/sec

counter.

Define degraded behavior beside the normal policy

For ordinary API fairness:

Redis available
→ global token bucket

Redis unavailable
→ local conservative bucket
→ emit degraded metric

For expensive report generation:

Redis unavailable
→ local concurrency cap
→ reject excess

For login attempts:

Redis unavailable
→ stricter local security threshold
or fail closed

For health checks:

bypass rate limiter

The failure mode becomes visible application policy rather than an accidental property buried in middleware.

A simplified wrapper might look like:

type RateLimitMode = "normal" | "degraded-local" | "fail-closed";

async function checkLimit(request: RequestContext): Promise<RateLimitDecision> {
  const policy = policies.forEndpoint(request.route);

  try {
    return await distributedLimiter.check({
      key: policy.key(request),
      limit: policy.limit,
      cost: policy.cost(request),
    });
  } catch (error) {
    metrics.rateLimiterBackendError.inc();

    switch (policy.failureMode) {
      case "fail-open":
        return {
          allowed: true,
          mode: "degraded-local",
        };

      case "local-fallback":
        return localLimiter.check(policy.key(request), policy.localLimit);

      case "fail-closed":
        return {
          allowed: false,
          mode: "fail-closed",
        };
    }
  }
}

The exact implementation is not the interesting part.

What matters is that failure semantics are explicit.

I would not want a system's admission-control policy to depend on a forgotten:

catch {
  return true;
}

whose rationale exists only in somebody's memory.

The Limiter Must Have a Smaller Blast Radius Than the System

A control mechanism should be proportionate to the thing it protects.

Suppose:

GET /health

takes:

1 ms

while consulting a distant global Redis cluster adds:

25 ms

The rate-limiting decision costs 25 times as much latency as the operation itself.

That is probably the wrong boundary.

Likewise, if five independently deployed regions all synchronously depend on one global Redis cluster for every request, the architecture has turned:

5 regional failure domains

into:

1 global limiter failure domain

That may be justified for a hard global security invariant.

For ordinary API fairness, it can be a terrible trade.

A reliability mechanism should not casually become the largest single point of failure in the system it was introduced to protect.

Cloudflare's localized approach is useful partly because it avoids paying global coordination cost on every request.

Regional budgets do something similar by moving coordination into a slower control plane.

Local fallback does it during failure.

These designs all accept some form of approximation or temporary inefficiency in exchange for keeping failures contained.

Sometimes technically wrong is operationally better

Suppose two regions briefly disagree.

Region A believes:

remaining = 3

while region B believes:

remaining = 1

Two requests slip through beyond the exact quota.

The limiter was technically wrong.

But:

service remained available
latency remained low
the protected dependency remained healthy
the customer noticed nothing

Now compare that with a strongly coordinated global design that blocks every request during a network partition.

Its counter may be mathematically correct.

Its service is unavailable.

Which architecture is more correct depends entirely on the invariant being protected.

For many fairness policies, bounded overshoot is the better failure mode.

For a security boundary, it may not be.

Distributed engineering is often about deciding which kind of wrongness the system can safely tolerate.

The Invariants I Want

For a distributed rate limiter, the guarantees I care about are closer to:

One API instance must not independently grant
the entire intended global quota.

Limiter state updates must not lose increments
because of local command races.

A limiter-backend outage must have an explicit
failure policy.

One limiter failure must not unintentionally take
down the entire API unless fail-closed behavior
is deliberate.

Security limits and fairness limits should not
automatically share the same failure semantics.

Expensive operations should be protected by
capacity controls tied to their actual resource use.

Approximation error should be measured as
false allows and false rejects.

Rate-limit metadata returned to a client should
describe the same decision that admitted or
rejected the request.

The limiter itself should expose latency,
errors, fallback activity, and rejection volume.

A globally exact quota should justify the latency,
availability, and blast-radius cost of global
coordination.

A hot tenant must not be able to turn the limiter
itself into the system bottleneck.

Degraded operation should preserve the underlying
invariant even if exact quota accounting is lost.

Those guarantees are much more useful than:

we use Redis INCR

because INCR solves the easy part.

The difficult part is deciding what the system should do when the state behind that increment is partial, stale, inaccessible, or expensive to coordinate.

Final Thought

Rate limiting looks like arithmetic:

count requests
compare with threshold
reject excess

Once the API is distributed, the interesting questions move elsewhere.

Where does the state live?

How current does it need to be?

How global is the quota?

Who owns the clock?

How much burst is acceptable?

How much inconsistency is tolerable?

What happens during Redis failover?

Can a hot tenant overload the limiter itself?

Can a replica return stale state?

Should login protection behave the same way as a free-tier quota?

Should an expensive endpoint be limited by requests per second or by concurrent work?

What happens when the mechanism whose job is to preserve availability becomes unavailable itself?

Those questions reveal what a rate limiter actually is.

It is not merely a counter.

It is an admission-control policy implemented by a distributed system.

Once we see it that way, perfect enforcement stops being the obvious goal.

For a security boundary, strong enforcement may justify sacrificing availability.

For API fairness, small approximation may be harmless.

For database protection, concurrency may matter more than request rate.

For multi-region systems, local independence may be more valuable than perfectly synchronized counters.

So the rate-limiting question I care about most is not:

“Which algorithm is most accurate?”

It is:

“What is the cost when this limiter is wrong—and what is the cost when the limiter itself is unavailable?”

Once those two costs are explicit, the architecture usually becomes much easier to choose.

References

• Stripe Engineering — Scaling your API with rate limiters. Describes production use of request-rate limiters, concurrent-request limiters, fleet-level shedding, token buckets, and Redis, including the importance of making limiter middleware resilient when Redis or the limiter itself fails.

• GitHub Engineering — How we scaled the GitHub API with a sharded, replicated rate limiter in Redis. Describes moving API rate-limit state into sharded and replicated Redis, using Lua for atomic operations, handling stale replica reads, and keeping rate-limit metadata consistent with the decision that produced it.

• Cloudflare — How we built rate limiting capable of scaling to millions of domains. Discusses the tradeoff between purely local counters and globally centralized state, and describes an approximate sliding-window design evaluated across hundreds of millions of requests.

• Redis Documentation — Redis rate limiter. Describes Redis as shared rate-limit state across service instances and covers atomic operations for fixed windows, sliding windows, and token buckets.

• Redis Documentation — INCR rate-limiter pattern. Documents fixed-window implementations, expiration races, transactions, and Lua-based atomic updates.

• Redis Documentation — INCREX. Documents the atomic increment primitive that can combine numeric updates, bounds, and expiration behavior in a single operation.