When Does a Cache Become a Reliability Dependency Instead of a Performance Optimization?

Caching usually begins with an innocent goal.

A database query takes 120 milliseconds, the result changes infrequently, and thousands of users request the same data. Put Redis in front of it and the request may take only a few milliseconds. Database load drops, infrastructure becomes cheaper, and the product feels faster.

Then six months later, Redis goes down.

The application falls back to the database exactly as designed—and the database collapses.

At that point, the cache was no longer merely a performance optimization. It had become part of the application's reliability architecture.

The reason is easy to miss because caching changes the workload seen by everything behind it. Suppose a database originally handled:

10,000 reads / second

and a cache eventually reaches:

95% hit rate

The database may now see only:

500 reads / second

Over time, infrastructure gets resized around that number. Traffic grows. Uncached queries become more expensive. Other workloads begin consuming the database capacity that caching freed.

Then the cache disappears and the backend is suddenly asked to absorb something much closer to the original 10,000 reads per second again—even though the architecture may no longer be capable of doing so. Amazon's Builders' Library describes this progression as a system becoming effectively addicted to its cache: a layer introduced to improve latency or reduce cost eventually becomes necessary to prevent the dependency behind it from being overwhelmed.

That leads to a much more useful production question than:

“Does caching make this endpoint faster?”

The question is:

“What happens to the rest of the system when the cache stops working?”

When the Cache Changes the Capacity Model

The classic cache-aside pattern looks harmless:

A simple implementation might look like:

async function getProduct(productId: string) {
  const cacheKey = `product:${productId}`;

  const cached = await redis.get(cacheKey);

  if (cached) {
    return JSON.parse(cached);
  }

  const product = await productsRepository.findById(productId);

  await redis.set(cacheKey, JSON.stringify(product), { EX: 300 });

  return product;
}

The design is attractive because Redis is not the source of truth. If a key disappears, the application can query the database, rebuild the cache entry, and continue.

The hidden assumption is that the database can absorb the miss traffic.

That assumption may stop being true long before anyone notices.

Suppose production receives:

20,000 reads / second

with a:

98% cache hit rate

Only about:

400 reads / second

reach the database.

Now remove the cache:

20,000 / 400 = 50×

The database sees a fifty-fold increase even though user traffic has not changed.

The architecture itself created the load multiplier.

That is why:

we can just fall back to the database

is not a complete failure strategy.

The fallback path needs a capacity model of its own.

Test the System Without the Cache

One of the strongest tests for a production cache is also one of the simplest:

Turn it off.

Amazon explicitly recommends testing services with caching disabled so teams can verify that safeguards such as downstream request caps and load shedding actually protect the backing service during cache loss.

A dashboard saying:

Redis hit rate = 97%

is useful during normal operation.

The more important reliability question may be:

At 0% hit rate, how much production traffic
can the database safely absorb?

Suppose the answer is:

25%

Then Redis is operationally critical whether or not the architecture diagram calls it "just a cache."

That does not automatically make the design bad. At sufficiently large scale, provisioning the persistent datastore for every request under complete cache loss may be economically or technically unreasonable.

The mistake is not relying on caching.

The mistake is discovering during an outage that you were relying on it all along.

Cache Misses Are Correlated, Not Independent

A cache outage does not necessarily create one database query for every missing object.

For popular objects, it can create thousands of identical queries at once.

Suppose:

product:iphone

is extremely hot and its cached value expires. At almost the same instant:

2,000 requests

ask for it.

Each request does:

const cached = await redis.get(key);

if (!cached) {
  return database.loadProduct(id);
}

Every caller sees:

MISS

and every caller asks the database for the same object.

This is the cache stampede, or thundering-herd problem. Both Amazon and Redis discuss coordinating refresh work so one missing popular value does not turn into a large burst of identical downstream operations.

Coalesce identical misses

Instead of:

2,000 misses
→ 2,000 database queries

we want something closer to:

2,000 misses
→ 1 database query
→ 1,999 requests share the result

Within one Node.js process, an in-flight promise map can provide a simple form of request coalescing:

const inflight = new Map<string, Promise<unknown>>();

async function cached<T>(key: string, loader: () => Promise<T>): Promise<T> {
  const hit = await redis.get(key);

  if (hit !== null) {
    return JSON.parse(hit) as T;
  }

  const existing = inflight.get(key);

  if (existing) {
    return existing as Promise<T>;
  }

  const promise = loader()
    .then(async (value) => {
      await redis.set(key, JSON.stringify(value), { EX: 300 });

      return value;
    })
    .finally(() => {
      inflight.delete(key);
    });

  inflight.set(key, promise);

  return promise;
}

Now identical misses inside that process share one refresh.

Across a distributed application fleet, the problem is harder because multiple instances can independently decide to refresh the same value. Options include distributed locking, cache-native coordination, probabilistic refresh, dedicated refresh workers, or simply allowing a bounded amount of duplicate work.

There is no universally correct implementation.

The important realization is that miss concurrency matters just as much as miss percentage.

Add jitter to expiration

Expiration itself can synchronize traffic.

Imagine 100,000 entries are warmed at approximately the same time with:

await redis.set(key, value, {
  EX: 300,
});

Five minutes later, a huge portion of the keyspace becomes eligible to expire together.

A little randomness spreads that regeneration work:

function ttlWithJitter(baseSeconds: number, jitterSeconds: number) {
  return baseSeconds + Math.floor(Math.random() * jitterSeconds);
}

await redis.set(key, value, {
  EX: ttlWithJitter(300, 60),
});

Entries now expire across approximately:

300–360 seconds

instead of converging on one moment. Redis recommends TTL jitter as one mitigation against expiration-driven stampedes.

This is the same principle behind retry jitter: large distributed populations behaving identically at exactly the same time can turn reasonable local behavior into a system-level event.

TTLs and Invalidation Are Consistency Decisions

TTL is often selected casually:

Five minutes sounds reasonable.

But expiration is not merely a Redis configuration.

It encodes a product guarantee.

A five-minute TTL means the system accepts that, under normal conditions, a client may see data several minutes behind the source of truth.

For something like:

country list
product description
configuration metadata

that might be perfectly acceptable.

For:

account permissions
inventory
payment state
risk decisions

it may not be.

The real question is:

How stale may this value safely become?

Amazon's caching guidance makes the same point: expiration policy should follow how frequently the source changes and how much staleness consumers can tolerate.

Invalidation is a race, not just a command

Consider the familiar update pattern:

await database.updateProduct(productId, update);

await redis.del(`product:${productId}`);

Now introduce concurrency:

Request A reads the old database value.

Request B updates the database.

Request B deletes the cache entry.

Request A writes its old value into Redis.

Every local operation looked reasonable.

The resulting cache is still wrong.

Meta has written extensively about this class of problem. Dynamic cache state changes on both the read path through cache fills and on the write path through invalidation, creating races that can become user-visible consistency failures.

That is why:

cache invalidation

is not just a clever old joke.

It is a concurrency problem.

Eventual consistency still needs a product definition

An eventually consistent cache is not permission to serve arbitrary old data.

The useful question is:

Which inconsistencies can the product tolerate?

A follower count lagging briefly may be acceptable.

A revoked administrator permission remaining active may not be.

A stale inventory count during a scarce-product checkout may produce incorrect business behavior.

The caching strategy should follow the consequence of being wrong, not simply the ease of putting a value in Redis.

Stale Data Can Be a Reliability Feature

Now consider a different failure mode.

Redis still contains:

product price = $49

but the value has reached the age at which the application would normally refresh it. The database is currently unhealthy.

Should the service throw away the cached value and force the request through the failing dependency?

Sometimes the safer choice is to serve the last known good value for a little longer.

Amazon describes using soft and hard TTLs in some systems:

soft TTL:
attempt refresh

hard TTL:
do not serve beyond this point

If refresh fails after the soft TTL but before the hard TTL, the application can continue serving stale data temporarily. That gives the dependency time to recover instead of making every request join the failure.

For example:

interface CacheEnvelope<T> {
  value: T;
  refreshedAt: number;
  hardExpiresAt: number;
}

Then:

async function getConfig(): Promise<Config> {
  const cached = await loadCache<Config>("config");

  if (!cached) {
    return refreshConfig();
  }

  const age = Date.now() - cached.refreshedAt;

  if (age < SOFT_TTL_MS) {
    return cached.value;
  }

  try {
    return await refreshConfig();
  } catch {
    if (Date.now() < cached.hardExpiresAt) {
      return cached.value;
    }

    throw new Error("Config unavailable and stale limit exceeded");
  }
}

The architecture is making a reliability trade:

slightly stale response

versus:

no response

For configuration, catalogs, or other slowly changing data, stale may be dramatically better than unavailable.

For permissions, balances, or other correctness-sensitive state, the opposite may be true.

Stale serving is useful only when the application has explicitly defined the maximum acceptable staleness.

Cache Failures Need Bounded Fallback

Suppose Redis is unavailable.

A naïve cache-aside fallback does this:

async function getProduct(id: string) {
  try {
    const cached = await redis.get(`product:${id}`);

    if (cached) {
      return JSON.parse(cached);
    }
  } catch {
    // Cache unavailable. Continue.
  }

  return products.findById(id);
}

For one request, this looks resilient.

Across every application instance simultaneously:

Redis outage
→ every request bypasses cache
→ every request reaches database
→ database overload

The system has traded one failed dependency for two.

A safer design may cap fallback concurrency:

import pLimit from "p-limit";

const databaseFallback = pLimit(100);

async function loadWithoutCache(id: string) {
  return databaseFallback(() => products.findById(id));
}

Depending on the workload, the protection could instead be a rate limiter, load shedding, stale serving, or another explicit admission-control mechanism.

The principle is more important than the implementation:

Cache failure must not imply unlimited access to the backing store.

Amazon explicitly recommends bounding downstream calls or shedding work during extended cache failures rather than allowing fallback behavior to brown out the backing dependency.

Negative caching protects the miss path too

Not every expensive miss represents an object that exists.

Suppose clients repeatedly request:

/product/does-not-exist

Without negative caching:

Redis miss
→ database query
→ not found

happens every time.

A buggy or malicious client can generate large numbers of random nonexistent identifiers and bypass the value cache entirely.

One mitigation is caching the negative result briefly:

const NOT_FOUND = "__NOT_FOUND__";

async function getProduct(id: string) {
  const key = `product:${id}`;

  const cached = await redis.get(key);

  if (cached === NOT_FOUND) {
    return null;
  }

  if (cached) {
    return JSON.parse(cached);
  }

  const product = await products.findById(id);

  if (!product) {
    await redis.set(key, NOT_FOUND, { EX: 30 });

    return null;
  }

  await redis.set(key, JSON.stringify(product), { EX: 300 });

  return product;
}

The negative TTL can be shorter because "not found" may eventually become "exists."

The broader idea is important: failures and absences can be cacheable outcomes too when doing so protects an expensive dependency from repetitive work.

Redis Has Its Own Capacity Limits

Redis is fast.

It is not infinite.

A cache cluster still has finite:

CPU
memory
network bandwidth
connections
per-shard capacity

and workload distribution matters.

Suppose 99% of keys distribute evenly, but one key receives 30% of all traffic:

homepage:trending

That can create a hot-key problem:

Cluster-wide CPU may look healthy while one shard is saturated.

Redis has added tooling aimed specifically at identifying disproportionately expensive keys, including HOTKEYS reporting based on CPU and network use.

The lesson should look familiar:

Aggregate capacity does not protect a system from skew.

Eviction can turn memory pressure into database pressure

Once Redis approaches its configured memory limit, eviction policy determines what happens.

Depending on configuration, Redis can evict values according to LRU, LFU, randomness, TTL eligibility, or reject additional writes.

Operationally, that memory decision can become a database-load decision.

Suppose a hot working set normally fits in cache, but a traffic change causes useful entries to be evicted.

Hit rate falls:

98%
→ 94%
→ 85%
→ 70%

At:

20,000 requests / second

the miss traffic looks roughly like:

98% hit rate →   400 DB reads/s

90% hit rate → 2,000 DB reads/s

70% hit rate → 6,000 DB reads/s

A seemingly modest deterioration in cache effectiveness can multiply database load dramatically.

Monitoring Redis memory without connecting it to hit rate, eviction rate, and downstream query volume therefore misses the system-level consequence.

A Falling Hit Rate Is a Downstream Load Problem

This dashboard entry:

cache hit rate = 93%

does not tell me whether the system is safe.

I would rather see:

request rate:          40,000/s
cache hit rate:            93%
cache miss rate:             7%
database fallback:        2,800/s
database safe capacity:   4,000/s

Now we know the margin.

If hit rate falls to:

88%

database traffic becomes approximately:

4,800/s

which exceeds the known safe capacity.

That means the meaningful alert may not be:

Redis unavailable

It may be:

cache hit rate < 90%

because that is where the backing service becomes endangered.

This reframes cache monitoring.

Hit rate is not only a cache-efficiency metric.

It is an input into the capacity model of every system behind the cache.

The useful operational question is:

At the current request rate,
what does this miss percentage mean
for the backing service?

That connection turns a cache dashboard into a reliability dashboard.

Local Caches Add Resilience and Complexity

One way to reduce dependence on a shared external cache is to add another layer:

The normal hierarchy becomes:

local hit
→ fastest

then:

Redis hit
→ shared cache

then:

database

If Redis becomes unavailable, a warm local cache may preserve both latency and backend protection for some portion of the workload. Amazon describes local caching alongside an external cache as one way to retain useful data during external-cache failures.

The tradeoff is that each process now holds an independent replica:

Instance A cache
≠
Instance B cache

Clients routed between instances may observe different values. Invalidation becomes more difficult. Memory use is duplicated. Deployments may leave each process with a different cache population.

Every additional cache layer protects some failure modes while introducing new consistency questions.

That is why I find this mental model useful:

A cache is another replica of state.

Once you think of it that way, familiar distributed-systems questions appear naturally:

How is the replica populated?

How is it invalidated?

How stale may it become?

What happens when updates arrive out of order?

How does it recover after failure?

Which copy is authoritative?

Meta's work on Memcache and TAO demonstrates how significant those questions become at large scale, particularly when replication lag and cache invalidation interact across regions.

Redis may provide fast key-value access.

The architecture around it is still a distributed-state system.

Cache Formats Are Deployment Contracts

Caches can also complicate otherwise safe rolling deployments.

Suppose version 1 writes:

{
  "id": "123",
  "name": "Alice"
}

while version 2 expects:

{
  "id": "123",
  "displayName": "Alice",
  "version": 2
}

During deployment:

v1 instances
+
v2 instances

may share the same Redis cluster.

Each version can therefore read data written by the other.

If v2 assumes every cached object contains:

displayName

then an old cached representation can break new code.

Amazon recommends treating external cache formats similarly to persistent formats during mixed-version deployments: new versions should tolerate older values, older versions should tolerate compatible additions, and engineers should be careful about solving incompatibility by flushing everything at once because a mass refresh can itself overwhelm downstream systems.

One option is explicit versioning:

interface CachedProductV2 {
  version: 2;
  id: string;
  displayName: string;
}

Cache keys can also encode schema versions:

product:v1:123
product:v2:123

That temporarily duplicates storage but keeps representations unambiguous.

Another option is tolerant deserialization:

function parseCachedProduct(raw: string): Product {
  const value = JSON.parse(raw);

  if (value.version === 2) {
    return fromV2(value);
  }

  return fromLegacy(value);
}

The key lesson is:

cache is disposable

does not imply:

cache format changes are operationally irrelevant

A cache may be disposable in theory while being too large or too important to discard instantaneously in production.

The expand-and-contract mindset used for database schemas applies surprisingly well here too.

Choose What to Cache Deliberately

Caching can hide poor underlying behavior for a long time.

Suppose a database query takes:

1.5 seconds

but the endpoint maintains:

99.9% cache hit rate

Almost nobody experiences the uncached path.

That makes it easy for the raw query to remain inefficient for years—until a cache failure suddenly exposes it to every request.

Periodically measuring:

uncached latency

is therefore a reliability exercise, not just a database optimization exercise. The cache should not prevent the team from understanding the performance characteristics of the system underneath it.

Caching also depends on traffic shape.

If:

80% of requests

target:

20% of objects

the cache may work extremely well.

If product behavior changes and traffic becomes more uniformly distributed, the same cache capacity may suddenly produce many more misses. Amazon recommends validating cache sizing and expiration behavior against real traffic distributions because workload popularity can invalidate assumptions that were previously correct.

Before caching something, I would ask:

Is there meaningful reuse?

If nearly every request asks for a unique value once, the cache may add:

network hop
serialization
memory use
operational complexity

without producing enough hits to justify itself.

Can the data be stale?

If the answer is no, caching becomes substantially harder.

Is regeneration expensive?

A cache provides more value when the miss path is expensive.

Can miss traffic be bounded?

If cache loss can collapse the source of truth, the fallback path needs an explicit protection mechanism.

What happens when the cache is wrong?

An old profile picture and an outdated permission are not equivalent failures.

Is Redis even the right layer?

Sometimes an:

HTTP cache
CDN
process-local cache
browser cache
database-level optimization

solves the problem with fewer moving parts.

Caching should solve a measurable problem.

It should not exist simply because high-performance architectures are expected to have Redis.

Different cache strategies move the reliability boundary

Cache-aside:

read cache
miss → read DB
populate cache

keeps the cache outside the authoritative write path, but introduces miss races and stale-fill races.

Write-through:

write
→ caching layer
→ persistence

can keep hot values aligned more directly, but cache availability now participates in writes.

Write-behind:

write cache
→ acknowledge
→ persist later

can reduce latency significantly but temporarily makes the caching layer authoritative state, which changes durability semantics.

Invalidate-on-write:

write database
→ remove cache entry

is common and relatively simple, but ordering and concurrent cache fills still need careful treatment.

There is no universally superior strategy because each moves the consistency and availability tradeoffs somewhere different.

Observe the Cache as a System

At minimum, I want cache-level telemetry such as:

hit rate
miss rate
latency
errors
memory usage
evictions
connections
CPU
network throughput

But those metrics are incomplete unless they are connected to the systems the cache protects.

I also want:

database requests caused by misses
database latency during miss spikes
refresh count
coalesced request count
hot-key distribution
stale responses served
fallback requests rejected

A useful dashboard could tell a story like this:

Redis hit rate:              97.8%
Redis p99:                    3 ms
Evictions:                    0/s

Miss-driven DB queries:       440/s
DB safe capacity:           3,000/s

Projected DB load
at 90% hit rate:            2,000/s

Projected DB load
at cache loss:             20,000/s

The last number is particularly important.

It tells you whether the cache is merely accelerating an otherwise healthy path or actively preventing the backing system from receiving a workload it cannot survive.

That distinction should influence alerting, capacity planning, incident response, and how seriously the cache itself is operated.

Test Failure Modes, Not Just Hit Latency

Benchmarking this:

GET key = 0.7 ms

tells you that Redis is fast.

It says almost nothing about whether the caching architecture is resilient.

A more useful failure matrix is:

The original source makes exactly this distinction: deliberate cache-failure testing reveals far more about production reliability than measuring a fast cache hit.

The tests should answer how the whole system behaves as caching degrades.

That includes partial failure.

A slow Redis cluster may be more dangerous than an obviously dead one if every request waits on the cache timeout before eventually falling through to the database.

A 90% hit rate may be more operationally interesting than a 0% hit rate if 90% is precisely where database capacity crosses its safe limit.

Reliability testing should therefore cover gradual degradation as well as clean outages.

A Safer Cache-Aside Architecture

A more defensive cache-aside path might look like this:

The interesting component in that diagram is not Redis.

It is the behavior around Redis.

The design explicitly answers:

Who is allowed to refresh a hot key?

How many fallback database operations
can run at once?

Can the application serve stale data?

How stale?

What happens when no downstream
capacity remains?

What happens if Redis itself is slow?

What state is safe to cache negatively?

Those are reliability decisions.

The cache client API is implementation detail.

When a Cache Becomes a Reliability Dependency

There is a simple thought experiment.

Disable the cache.

If the result is:

latency increases

while the service remains stable, the cache is primarily a performance optimization.

If the result is:

database overload
timeouts
retry amplification
cascading failures
service outage

then the cache is part of the reliability architecture.

The source makes an important qualification here: this is not automatically a design flaw. At sufficiently large scale, expecting persistent storage to absorb the entire uncached workload may be unrealistic. Large systems such as Meta's have relied heavily on caching layers precisely because sending full read volume directly to persistent storage would be impractical.

The mistake is not dependency.

It is accidental dependency.

Once the architecture requires the cache for survival, that dependency deserves to be understood and operated explicitly.

You should know:

What happens if it fails?

What happens if it starts empty?

What happens when traffic distribution changes?

What happens when one key becomes hot?

What happens when memory fills?

How stale may cached state become?

What is the recovery path?

Can the database survive that recovery?

Those are the same kinds of questions we ask about databases, queues, external services, and other systems capable of taking production down.

The cache has earned the same level of attention.

The Invariants I Want

For a production cache, the guarantees I care about are closer to:

One hot-key expiration must not create thousands
of identical downstream queries.

Loss of Redis must not result in unbounded
database traffic.

Large portions of the keyspace should not expire
in a synchronized wave.

The application must explicitly define how stale
each important class of data may become.

Invalidation behavior must preserve the
correctness requirements of the product.

A falling hit rate should be observable before
the database reaches saturation.

Memory pressure and eviction must not silently
turn into uncontrolled backend load.

Mixed application versions must be able to
coexist with cached representations.

Missing or failing downstream results should not
cause unlimited repeated lookups.

The system must know what happens at
0% cache hit rate.

Those guarantees are much more useful than:

Redis is highly available.

because Redis availability is only one part of cache reliability.

A perfectly available Redis cluster can still produce stale state, hot-key saturation, synchronized expiration, excessive eviction, dangerous miss traffic, or a deployment compatibility failure.

The invariant belongs to the complete system.

Final Thought

Caches are seductive because they make systems look healthier.

Latency falls. Database CPU falls. Infrastructure cost falls. Throughput increases.

Then the architecture begins adapting to those improved numbers.

Eventually:

the database does not need
to handle full traffic

quietly becomes:

the database cannot handle
full traffic

and a performance optimization has become an architectural dependency.

That is not necessarily wrong. Large systems cannot always keep every backing layer permanently provisioned for the full workload that caching normally absorbs.

But the dependency should be understood.

Once a cache becomes essential, it deserves the same operational questions we ask of anything else keeping production alive: how it fails, how it recovers, what load moves elsewhere during that failure, which guarantees weaken, and what prevents one degraded component from taking another one down with it.

The most important caching metric may therefore not be:

“What is our hit rate?”

It may be:

“What happens if that hit rate suddenly becomes zero?”

If you cannot answer that confidently, Redis is no longer just making the system faster.

It is helping keep the system alive.

And it should be designed, tested, monitored, and operated accordingly.

References

• Amazon Builders' Library — Caching challenges and strategies. A production treatment of cache dependency, cold caches, cache stampedes, stale serving, soft and hard TTLs, negative caching, request coalescing, cache-format compatibility, and protecting backing services during cache outages.

• Meta Engineering — Cache made consistent. Explains why dynamic distributed caches create difficult consistency races and how Meta improved consistency in systems including TAO and Memcache.

• Meta Engineering — Scaling memcache at Facebook. Describes the role large-scale caching plays in protecting database infrastructure while serving very high read volumes.

• Meta Engineering — TAO: The power of the graph. Describes Meta's multi-tier cache architecture, eventual-consistency tradeoffs, persistent-storage protection, and heavily skewed social-graph workloads.

• Redis — Using caches, promises, and locks. Discusses cache stampedes and coordinating refresh work so concurrent misses do not generate identical expensive operations against the backing store.

• Redis Documentation — Key eviction. Documents maxmemory, LRU, LFU, and other eviction policies and how Redis behaves as memory reaches its configured limit.

• Redis Documentation — HOTKEYS. Documents tooling for detecting keys that disproportionately consume Redis CPU or network bandwidth.