paulserban.eu

Writing Edition

Paul Serban

Writing, snippets & book notes

Vol. 2026

January 21, 2022

Caching Deep Dive: Mechanisms, Strategies, and Systems for Fullstack Engineers

A practical, end-to-end guide to how caching actually works across the browser, CDN, application, and database layers

Introduction

Caching is one of the few techniques in software engineering that pays off at nearly every layer of a system, from the browser tab a user has open to the storage engine underneath a database. Done well, it can turn a 200ms database round trip into a 2ms memory lookup, cut infrastructure costs by an order of magnitude, and make an application feel instantaneous. Done poorly, it introduces the two hardest problems in computer science made worse: cache invalidation and consistency, now distributed across multiple failure domains.

This article is a deep dive into caching as it actually shows up in fullstack engineering: not just "use Redis", but the mechanisms underneath HTTP caching, browser storage, CDN edge caches, application-level caches, and database caching, along with the strategies (cache-aside, write-through, write-behind, read-through) and eviction policies (LRU, LFU, TTL) that govern how they behave under load. The goal is to leave you with a mental model precise enough to reason about a cache you didn't design, and confident enough to design one you will have to defend in a design review.

Why Caching Exists: The Problem Overview

Every cache exists to close a gap between two things: how fast you need data and how fast your source of truth can produce it. A relational database enforcing ACID guarantees, doing disk I/O, and executing query plans is fundamentally slower than reading a value out of RAM. A server in Virginia is fundamentally farther, in network terms, from a user in Singapore than an edge node in the same city. Caching doesn't eliminate this gap; it exploits locality and redundancy to make the gap irrelevant for the common case.

The underlying assumption that makes caching work is the Pareto-like skew present in almost all real workloads: a small set of keys, pages, or queries accounts for a large share of requests. This is sometimes called the 80/20 rule of access patterns, and it's why a cache with a fraction of the storage capacity of your primary data store can still absorb the majority of read traffic. If your access pattern were perfectly uniform, with every key touched equally often, caching would provide little benefit, because you'd need to cache nearly everything to get a meaningful hit rate.

The cost of getting this wrong is asymmetric. A cache miss is rarely catastrophic; it just falls through to the slower path you'd be using anyway without a cache. A stale cache hit, on the other hand, can silently serve wrong data: an out-of-stock item that appears available, a price that changed an hour ago, a permission that was revoked but is still being honored. This is why most of the engineering effort in caching systems goes not into the read path, which is usually trivial, but into invalidation, consistency, and the failure modes when the cache and the source of truth disagree.

The Layers of Caching: A Technical Deep Dive

Browser and HTTP Caching

The outermost cache in a web application is the browser itself, governed almost entirely by HTTP headers defined in RFC 7234. When a browser requests a resource, the server can attach Cache-Control directives that dictate how long the response may be reused without revalidation. Cache-Control: max-age=3600 tells the browser it can serve the cached copy for an hour without asking the server anything. Cache-Control: no-cache (a somewhat confusingly named directive) means the browser must revalidate with the server on every use, even though it may still store the response. Cache-Control: no-store forbids caching entirely, which matters for responses containing sensitive data. A separate private versus public directive controls whether intermediate shared caches, such as a CDN or corporate proxy, are allowed to store the response at all, or whether it may only be cached in the end user's own browser; a response containing per-user data like an account balance should be marked private so a CDN doesn't accidentally serve it to a different user.

It's worth being precise about what "the browser cache" actually is, because it isn't one store. Chromium-based browsers and Firefox maintain a memory cache for the current tab session (fastest, but cleared when the tab or process closes), a disk cache that persists across restarts and is what most Cache-Control directives are really governing, and, separately, a back/forward cache (bfcache) that snapshots an entire page's JavaScript execution state so that pressing the browser's back button can restore the page instantly without re-running any code. The bfcache is disabled by certain page behaviors, most notably an unclosed WebSocket connection, use of the Cache-Control: no-store directive on the main document, or listeners on the unload event, which is one of several reasons unload has been deprecated in favor of pagehide and visibilitychange. Engineers debugging "why does going back feel slow" often discover the culprit is one of these bfcache-blocking patterns rather than anything related to Cache-Control on subresources.

Revalidation itself is cheap because of conditional requests. A server can return an ETag, an opaque fingerprint of the resource's content, or a Last-Modified timestamp. On the next request, the browser sends If-None-Match or If-Modified-Since, and if the resource hasn't changed, the server responds with a 304 Not Modified and an empty body, saving the bandwidth of retransmitting content the client already has. This is the mechanism that lets a cache stay both fast and correct: it costs a round trip, but not a full payload transfer. ETag is generally preferred over Last-Modified because it can detect content changes at sub-second granularity and doesn't depend on filesystem timestamps surviving a deployment, but the two can be used together, with the server accepting either conditional header.

When no explicit Cache-Control or Expires header is present, browsers don't treat the response as uncacheable by default; RFC 7234 defines heuristic freshness, under which a browser may estimate a reasonable cache lifetime from the Last-Modified timestamp, typically around ten percent of the age of the resource at the time it was fetched. This is a common source of confusing, hard-to-reproduce bugs: a resource served without explicit cache headers can still be cached, just according to a heuristic that varies by browser and isn't visible in the response itself. The practical lesson is to never rely on the absence of caching headers to mean "not cached"; if a response must always be fresh, say so explicitly with Cache-Control: no-cache or no-store rather than assuming the default is safe.

The stale-while-revalidate directive, standardized as an extension in RFC 5861 and now widely supported, addresses a real limitation of a hard max-age cutoff: without it, the request that happens to arrive right after expiration pays the full latency of a synchronous revalidation. Cache-Control: max-age=60, stale-while-revalidate=3600 tells the browser (and CDNs that support it) that for up to an hour past expiration, it's acceptable to serve the stale response immediately while revalidating in the background, so the user experiences no added latency and the next request gets the fresh copy. This pattern is especially valuable for content that changes infrequently but where synchronous revalidation delay would be noticeable, such as a navigation menu or a feature-flag configuration document.

A second, often overlooked HTTP caching lever is content-addressed URLs, commonly seen as app.a1b2c3.js instead of app.js. Because the filename changes whenever the content changes, you can set an extremely long max-age (a year, effectively "cache forever") with no risk of staleness, since a new deployment produces a new URL rather than overwriting the old one. This pattern, standard in tools like Webpack, Vite, and Next.js, sidesteps the invalidation problem entirely for static assets by making the URL itself the cache key and the content hash the version. The HTML document that references these hashed assets is typically served with a short TTL or no-cache, so that a deployment is picked up quickly, while everything the HTML links to is cached aggressively; this asymmetry, short TTL on the entry point and long TTL on everything it references, is the standard shape of a well-configured static site's caching policy.

Beyond passive HTTP caching, the Service Worker API gives a page programmatic control over caching through the Cache API, letting a script intercept every network request via a fetch event listener and decide, in code, whether to serve from cache, from network, or some combination of both. This is what powers offline-capable web applications and precise strategies like "cache-first for static assets, network-first for API calls" that HTTP headers alone can't express, because HTTP caching is declarative per-response while a service worker is imperative per-request. The trade-off is complexity and a well-known set of debugging headaches, particularly around service worker update lifecycles, where a stale service worker can keep serving an old version of an application indefinitely until the user closes every tab, which is why most service worker implementations pair a cache-first strategy for assets with an explicit versioned cache name that gets deleted on activation of a new worker.

// A representative service worker caching strategy: cache-first for
// static assets (versioned, so old caches are cleaned up on deploy),
// network-first for API calls (freshness matters more than speed).
const CACHE_NAME = "app-shell-v3";
const STATIC_ASSETS = ["/", "/app.a1b2c3.js", "/styles.f9e8d7.css"];

self.addEventListener("install", (event: ExtendableEvent) => {
  event.waitUntil(
    caches.open(CACHE_NAME).then((cache) => cache.addAll(STATIC_ASSETS))
  );
});

self.addEventListener("activate", (event: ExtendableEvent) => {
  // Delete any cache from a previous deploy that doesn't match the
  // current version, preventing indefinite staleness across releases.
  event.waitUntil(
    caches.keys().then((keys) =>
      Promise.all(
        keys.filter((key) => key !== CACHE_NAME).map((key) => caches.delete(key))
      )
    )
  );
});

self.addEventListener("fetch", (event: FetchEvent) => {
  const url = new URL(event.request.url);

  if (url.pathname.startsWith("/api/")) {
    // Network-first: always try the network; fall back to cache only
    // if the user is offline. Freshness matters more than speed here.
    event.respondWith(
      fetch(event.request).catch(() => caches.match(event.request))
    );
    return;
  }

  // Cache-first: static assets are content-hashed, so a cache hit is
  // always correct and we avoid a network round trip entirely.
  event.respondWith(
    caches.match(event.request).then((cached) => cached ?? fetch(event.request))
  );
});

CDN and Edge Caching

Content Delivery Networks extend HTTP caching semantics geographically. A CDN such as Cloudflare, Fastly, or Amazon CloudFront operates a fleet of edge servers, often called Points of Presence (PoPs), positioned close to end users via anycast routing and DNS-based or BGP-based traffic steering, each holding a copy of frequently requested content. The same Cache-Control headers that govern browser behavior also govern CDN behavior, though CDNs typically expose additional controls, like s-maxage, which applies specifically to shared caches (CDNs and proxies) and can differ from the max-age a browser respects, letting an operator cache a response at the edge for an hour while still telling browsers to revalidate every minute.

What makes CDN caching distinct from browser caching is the cache key and purge semantics. A CDN cache key is often more than the URL; it can include the Vary header's named request headers (such as Accept-Encoding or Accept-Language), cookies, or custom logic in an edge function. Getting the cache key wrong is one of the most common production incidents in caching: cache content meant to be personalized per-user but keyed only on URL will leak one user's data to another, a class of bug sometimes called cache deception or cache poisoning when an attacker deliberately manipulates unkeyed inputs, such as a header or query parameter the origin reads but the cache ignores, to get a malicious or sensitive response stored under a shared key. Conversely, over-including headers or cookies in the key can fragment the cache into thousands of near-duplicate entries with a near-zero hit rate, effectively defeating the purpose of caching at all. Because of this, most CDN platforms provide explicit tools for authoring the cache key: Cloudflare's Cache Rules, Fastly's VCL (vcl_hash subroutine), and CloudFront's cache policies all let an engineer declare exactly which headers, cookies, and query string parameters participate in the key, rather than leaving it to implicit defaults.

A second architectural detail worth understanding is tiered caching, sometimes called an origin shield. Rather than every edge PoP independently forwarding cache misses straight to the origin, a tiered setup routes misses from many edge nodes through a smaller number of regional "shield" caches first, so that a sudden spike in demand for a newly-cold object (a product just gone viral, a news article just published) results in one request to the origin per region rather than one request per edge node worldwide. Amazon CloudFront's "origin shield" and Fastly's shielding both implement this pattern directly, and it is one of the most effective, and most underused, levers for protecting an origin from the exact stampede behavior discussed later in this article, applied at the edge rather than inside the application.

CDNs also cache negative and error responses, deliberately, with their own TTLs distinct from successful responses. A well-configured CDN might cache a 404 Not Found for a short period (say, ten seconds) specifically to prevent a broken or misconfigured client that's retrying a bad URL in a loop from generating sustained load on the origin, while still recovering quickly once the underlying issue is fixed. The same logic applies to 5xx responses during an origin outage: some CDNs support serving a stale, already-cached copy of a resource even past its nominal expiration when the origin is unreachable, a feature usually called stale-if-error, which trades strict freshness for availability during an incident, often the correct trade-off when the alternative is showing users an error page for data that was perfectly fine a minute ago.

Purging is the other half of the CDN story. Because edge nodes are geographically distributed, invalidating a cached object means propagating a purge instruction to every node that might hold it, which is why most CDNs offer both instant single-URL purges and slower but broader "purge everything" or tag-based purges. Fastly and Cloudflare both support surrogate keys or cache tags, which let you associate multiple cached objects with a logical tag (for example, product:1234) and purge all of them in one call when that product changes, rather than tracking every URL variant individually. This matters because a single logical entity is frequently rendered into many distinct cached objects, a product page, a search result snippet, an API response, an Open Graph preview, and tag-based purging is the only practical way to invalidate all of them together without the application needing to enumerate every URL variant that might reference that product.

The rise of programmable edge compute, Cloudflare Workers, Fastly Compute, and AWS Lambda@Edge, has turned CDNs from passive header-obeying caches into a place to run real application logic, including custom caching decisions that HTTP headers alone can't express. The example below shows a Cloudflare Worker normalizing a cache key to strip marketing query parameters that don't affect the response, a common real-world source of cache fragmentation, since ?utm_source=twitter and ?utm_source=email are semantically identical pages that would otherwise be cached as separate objects.

// Cloudflare Worker: normalize the cache key so tracking parameters
// (utm_*, gclid, fbclid) don't fragment the cache into near-duplicate
// entries that each have to be fetched from origin independently.
export default {
  async fetch(request: Request): Promise<Response> {
    const url = new URL(request.url);
    const trackingParams = ["utm_source", "utm_medium", "utm_campaign", "gclid", "fbclid"];
    trackingParams.forEach((param) => url.searchParams.delete(param));

    const cacheKeyRequest = new Request(url.toString(), request);
    const cache = caches.default;

    let response = await cache.match(cacheKeyRequest);
    if (response) {
      return response;
    }

    response = await fetch(request);

    // Only cache successful, cacheable responses; never cache errors
    // or responses the origin explicitly marked private.
    if (response.ok && !response.headers.get("Cache-Control")?.includes("private")) {
      const cacheable = new Response(response.body, response);
      cacheable.headers.append("Cache-Control", "s-maxage=3600");
      await cache.put(cacheKeyRequest, cacheable.clone());
      return cacheable;
    }

    return response;
  },
};

Application-Level Caching

Inside your application tier, caching typically takes the form of an in-memory cache (a simple dictionary, an LRU cache library, or a process-local cache) or a shared, out-of-process cache like Redis or Memcached. The distinction matters enormously in a horizontally scaled system. An in-process cache is fast (no network hop) but incoherent across instances: instance A can have stale data while instance B has fresh data, and there's no single source of truth to invalidate against. A shared cache like Redis solves the coherence problem by centralizing state, at the cost of a network round trip and a new dependency that can itself become a bottleneck or a single point of failure if not run in a replicated, highly available configuration.

Redis and Memcached, while often discussed interchangeably, differ meaningfully. Memcached is a pure, multi-threaded key-value store with LRU eviction and no persistence or built-in data structures beyond byte strings; it is exceptionally simple and fast for the narrow job of caching. Redis is a superset in capability: it supports rich data structures (lists, sets, sorted sets, hashes, streams), optional persistence (RDB snapshots and AOF logs), pub/sub, and Lua scripting for atomic multi-step operations, which makes it useful not just as a cache but as a lightweight data store or message broker. The trade-off is that Redis's richer feature set and traditionally single-threaded execution model (Redis 6+ added I/O threading, but core command execution is still single-threaded) mean it can behave differently under contention than Memcached's simpler, more parallel design.

Many production systems don't pick between in-process and shared caching; they layer both, a pattern usually called a multi-tier or "near cache" architecture. The application first checks a small, short-TTL in-process cache (a package like lru-cache in Node.js, or Caffeine in the JVM world), and only on a miss there does it check the shared Redis or Memcached tier, falling through to the database only if both miss. The in-process tier absorbs the hottest fraction of traffic with zero network overhead, while the shared tier still provides cross-instance coherence for everything else, and a Redis outage degrades the system to "as slow as no shared cache" rather than a full outage, since the in-process layer keeps functioning independently. The cost is an added consistency subtlety: invalidating a key now means clearing it in the shared cache and, if you want strict correctness, somehow signaling every instance to drop it from its local tier too, typically through a pub/sub invalidation channel (Redis's PUBLISH/SUBSCRIBE, or a dedicated invalidation topic on Kafka) rather than a single delete call.

At scale, a single Redis or Memcached node runs out of memory or CPU headroom, which is where sharding and replication enter the picture. Redis Cluster partitions the keyspace across multiple nodes using a fixed set of 16,384 hash slots, assigning each key to a slot via CRC16 and each slot to a node, so that adding or removing nodes only requires rebalancing a bounded set of slots rather than rehashing the entire keyspace, a scheme closely related to consistent hashing. Memcached, having no native clustering, typically relies on client-side sharding, where the client library hashes each key to pick a target server from a static list; libraries implementing the "ketama" consistent-hashing algorithm are the de facto standard here, because naive modulo hashing would remap most keys to a different server every time the server list changes, causing a mass cache miss exactly when you're trying to scale. Replication, running one or more read replicas behind a primary, protects against node failure and can offload read traffic, but for a cache this is often less critical than for a primary database, since a lost cache node is, by definition, recoverable by re-populating from the source of truth; teams sometimes over-invest in cache high availability relative to the actual cost of a cold cache.

A related and frequently underestimated failure mode is the hot key problem: sharding distributes load evenly only when keys are accessed roughly evenly, but a single extremely popular key, a celebrity's profile, a viral post, a global configuration flag, can direct a disproportionate share of traffic to whichever one shard owns it, overwhelming that node while its siblings sit idle. Because this is a data-distribution problem rather than a capacity problem, adding more shards doesn't help; the standard mitigations are client-side replication of just the hot key across multiple cache nodes with a random selection on read, or promoting that specific key into the in-process tier described above so most reads never leave the application instance at all.

Two more practical concerns round out application-level caching in production. Serialization cost is easy to overlook: storing a large object as JSON is simple but pays a real CPU and payload-size cost on every read and write, and teams operating at high throughput often switch to a more compact binary format (Protocol Buffers, MessagePack) or apply compression for large values, trading a small amount of CPU for meaningfully less network and memory pressure. Cache warming, proactively populating a cache before it's needed rather than waiting for organic misses, matters most right after a deployment or a cache flush; a service that cold-starts into full production traffic with an empty cache can suffer a stampede against the database that looks identical to the incident scenario described earlier, which is why warming scripts that pre-load the known-hot keyset are standard practice for high-traffic caches.

Here is a realistic cache-aside implementation against Redis in TypeScript, including the two failure modes teams often miss: cache stampedes and negative caching.

import { createClient, RedisClientType } from "redis";

const redis: RedisClientType = createClient({ url: process.env.REDIS_URL });
await redis.connect();

const NULL_SENTINEL = "__NULL__";
const inFlight = new Map<string, Promise<unknown>>();

async function getUserProfile(userId: string): Promise<UserProfile | null> {
  const cacheKey = `user:profile:${userId}`;

  const cached = await redis.get(cacheKey);
  if (cached !== null) {
    return cached === NULL_SENTINEL ? null : JSON.parse(cached);
  }

  // Prevent a cache stampede: if 500 requests miss at once for the same key,
  // only one should hit the database; the rest await the same promise.
  if (inFlight.has(cacheKey)) {
    return inFlight.get(cacheKey) as Promise<UserProfile | null>;
  }

  const fetchPromise = (async () => {
    try {
      const profile = await db.users.findById(userId);

      // Negative caching: cache the "not found" result too, with a short TTL,
      // so repeated lookups for nonexistent users don't hammer the database.
      if (profile === null) {
        await redis.set(cacheKey, NULL_SENTINEL, { EX: 60 });
        return null;
      }

      await redis.set(cacheKey, JSON.stringify(profile), { EX: 3600 });
      return profile;
    } finally {
      inFlight.delete(cacheKey);
    }
  })();

  inFlight.set(cacheKey, fetchPromise);
  return fetchPromise;
}

This pattern, sometimes called request coalescing or "single-flight", is the difference between a cache that degrades gracefully when it expires and one that causes a thundering herd against your primary database the instant a popular key evicts.

Database and Query Caching

Databases cache internally, whether or not you build an application-level cache on top. PostgreSQL and MySQL both maintain a buffer pool (shared_buffers in Postgres, innodb_buffer_pool_size in MySQL/InnoDB) that keeps recently accessed pages in memory, avoiding disk reads on repeated access to the same rows. Query planners also cache execution plans to avoid re-parsing and re-optimizing identical query shapes on every execution; PostgreSQL's PREPARE statements and MySQL's prepared statement protocol both let a driver send a query once and reuse the parsed, optimized plan across many executions with different parameter values, which matters most for high-frequency, simple queries where planning overhead is a meaningful fraction of total execution time.

It's worth being explicit that this buffer pool caching happens beneath a second layer of caching most engineers don't think about directly: the operating system's own page cache. Even data evicted from a database's internal buffer pool is often still sitting in the OS's file system cache in RAM, so a "cache miss" from the database's perspective can still avoid an actual physical disk read. This is why database benchmarks run cold (right after a restart, with an empty OS page cache) look dramatically different from the same benchmark run warm, and why capacity planning that only accounts for the configured buffer pool size, ignoring the OS cache sitting underneath it, tends to underestimate how much RAM a database host is effectively using for caching in aggregate.

Indexes deserve a mention here too, because they function as a specialized form of caching even though they're rarely described that way. A B-tree index doesn't cache row data directly, but it caches a sorted, quickly-searchable structure that avoids the cost of a sequential scan, and the index's own pages are subject to the same buffer pool caching as table data, so a "hot" index is effectively double-cached: its structure avoids the need to scan, and its pages are usually memory-resident because they're accessed on nearly every query against that table. Read replicas occupy an adjacent but distinct role: they don't cache in the traditional sense (each replica holds a full, real copy of the data, not a subset), but they serve the same architectural purpose of absorbing read load away from a single node, and teams frequently combine read replicas with an application cache in front of them, accepting replication lag as one more source of the staleness budget discussed throughout this article.

Above the database engine itself, many teams introduce a materialized view or a denormalized read table as a form of caching: precomputing an expensive join or aggregation once, on a schedule or on write, so that reads never pay the cost of the computation. PostgreSQL's native materialized views (CREATE MATERIALIZED VIEW ... REFRESH MATERIALIZED VIEW) are a direct implementation of this idea, trading write-time or refresh-time cost for read-time speed. The standard REFRESH MATERIALIZED VIEW command rebuilds the entire view and, unless created WITH NO DATA and refreshed CONCURRENTLY, locks it against reads for the duration; the CONCURRENTLY option avoids the read lock at the cost of requiring a unique index on the view and taking somewhat longer to complete, a trade-off most production deployments accept. Extensions such as pg_ivm bring incremental materialized view maintenance to PostgreSQL, updating only the rows affected by a given write rather than recomputing the whole view, which is conceptually identical to event-based cache invalidation applied inside the database rather than in an external cache. This is conceptually the same trade-off as an application cache, just implemented inside the database rather than beside it.

It's also worth noting what database-level caching does not do for you: a common historical feature, MySQL's query cache, which stored the literal result set of a SELECT keyed on the exact query text, was removed entirely in MySQL 8.0 because it scaled poorly under write-heavy workloads (a single row change invalidated every cached query touching that table) and performed badly under concurrency due to a global lock. Its removal is a useful data point for a broader principle: a cache that can't invalidate precisely, at the granularity of what actually changed, tends to lose to one that can, even if the imprecise version is simpler to reason about at first glance. This is part of why the object-level or row-level caching typically done in an application tier, rather than caching whole query result sets inside the database, remains the dominant pattern for anything beyond read-heavy, rarely-written tables.

# A query-result cache decorator built on a materialized-view-style
# refresh: the cache key incorporates the query and its parameters, and
# a background refresh keeps hot queries warm without blocking readers.
import hashlib
import json
import time
from functools import wraps

def cached_query(ttl_seconds: int = 300):
    def decorator(fn):
        cache: dict[str, tuple[float, object]] = {}

        @wraps(fn)
        def wrapper(*args, **kwargs):
            raw_key = json.dumps({"args": args, "kwargs": kwargs}, sort_keys=True, default=str)
            cache_key = hashlib.sha256(raw_key.encode()).hexdigest()

            entry = cache.get(cache_key)
            now = time.monotonic()
            if entry is not None and now - entry[0] < ttl_seconds:
                return entry[1]

            result = fn(*args, **kwargs)
            cache[cache_key] = (now, result)
            return result

        return wrapper
    return decorator


@cached_query(ttl_seconds=60)
def top_selling_products(category_id: int, limit: int = 10):
    # Expensive aggregation query - a strong candidate for a materialized
    # view in the database, or a cache-aside layer, or both together.
    return db.execute(
        """
        SELECT product_id, SUM(quantity) AS units_sold
        FROM order_items
        WHERE category_id = %s
        GROUP BY product_id
        ORDER BY units_sold DESC
        LIMIT %s
        """,
        (category_id, limit),
    )

Implementation Patterns: Cache-Aside, Write-Through, and Write-Behind

The strategy you choose determines who is responsible for keeping the cache populated and who bears the latency cost of a miss. In cache-aside (also called lazy loading), the application code owns the logic: check the cache, and on a miss, read from the source of truth and populate the cache for next time, as shown in the Redis example above. This is the most common pattern because it's simple and resilient; if the cache is entirely unavailable, the application still functions, just slower, by falling through to the database on every request.

Cache-aside has a well-known race condition that's worth naming explicitly, because it causes real, hard-to-reproduce staleness bugs. Suppose request A reads a key, misses, and begins fetching the value from the database. Concurrently, request B writes a new value for that same key directly to the database and then invalidates the cache. If A's database read completed before B's write, but A's cache-populate write to the cache happens after B's invalidation, A will overwrite the cache with the stale value it fetched earlier, and the cache is now permanently wrong until the next write or TTL expiration. There is no purely local fix for this in the general case; the standard mitigations are a short TTL as a self-healing backstop (the strategy discussed throughout this article), a version or timestamp check before writing to the cache so a write can detect it's about to overwrite a newer invalidation, or, in especially write-heavy systems, moving to a strategy that avoids the race by construction, such as write-through.

A closely related and frequently debated decision is whether a write should update the cache with the new value or simply delete the cached entry and let the next read repopulate it. Deleting is usually the safer default: it sidesteps the exact race described above (there's no "stale write" to overwrite, because there's nothing in the cache to overwrite until a fresh read populates it), at the cost of one extra cache miss immediately after every write. Directly updating the cache on write avoids that miss but reintroduces the race condition and requires the write path to compute or have on hand the exact value that should now be cached, which isn't always trivial when the cached value is a computed aggregation or a joined view rather than a straightforward mirror of a single row. Meta's well-known engineering writeups on their memcache deployment (serving Facebook's web tier) documented exactly this trade-off at large scale and settled on invalidation-by-deletion as the default for most of their cached data, precisely because of this race condition.

Write-through caching flips the responsibility on writes: every write goes to the cache first (or simultaneously to the cache and the source of truth), and the cache guarantees it is never stale relative to the data it holds, because nothing is written without also updating the cache. This eliminates the class of bugs where a cache is out of sync after a write, but it makes every write pay the latency cost of updating two systems, and it does nothing for cold keys that were never written through the cache in the first place, since write-through only populates the cache as a side effect of writes, not reads. A subtler operational question write-through forces you to answer is ordering: should the cache write happen before, after, or atomically alongside the database write? Writing to the cache first and the database second risks a cache that's ahead of the source of truth if the database write then fails; writing to the database first and the cache second (the more common choice) risks a brief window where a concurrent reader sees the old cached value even though the database has already committed the new one, a narrower and generally more tolerable version of the same staleness problem every caching strategy has to accept somewhere.

Write-behind (or write-back) caching decouples the write from the source of truth entirely: writes land in the cache immediately and are asynchronously flushed to the database later, often batched. This is the fastest write path available, and it's used in systems where write throughput is critical and some risk of data loss on a cache crash is acceptable, such as high-frequency counters or session state. The risk is real, though: if the cache node dies before the flush happens, that write is gone, so write-behind is inappropriate for data where durability guarantees matter, like financial transactions. Batching is what makes write-behind attractive beyond just latency: instead of issuing one database write per user action, a write-behind layer can accumulate many updates to the same or different keys over a short window (say, one second) and flush them as a single batched statement, dramatically reducing the number of round trips and transactions the database has to process, which is exactly the pattern behind view-count or like-count counters on high-traffic content, where exact real-time precision in the database matters far less than not falling over under load.

# A write-behind counter buffer: increments are applied in-memory (or in
# Redis) immediately for fast reads, and flushed to the database in
# batches on a timer, trading some durability risk for write throughput.
import asyncio
from collections import defaultdict

class WriteBehindCounters:
    def __init__(self, db, flush_interval_seconds: float = 1.0):
        self._db = db
        self._pending: dict[int, int] = defaultdict(int)
        self._flush_interval = flush_interval_seconds
        self._lock = asyncio.Lock()

    def increment(self, post_id: int, amount: int = 1) -> None:
        # Fast path: no I/O, no lock contention with the flush loop
        # except the brief moment it swaps the buffer below.
        self._pending[post_id] += amount

    async def run_flush_loop(self) -> None:
        while True:
            await asyncio.sleep(self._flush_interval)
            async with self._lock:
                if not self._pending:
                    continue
                batch = dict(self._pending)
                self._pending.clear()

            # One round trip for the whole batch, rather than one
            # UPDATE per increment - this is the throughput win.
            await self._db.execute_many(
                "UPDATE posts SET view_count = view_count + %s WHERE id = %s",
                [(delta, post_id) for post_id, delta in batch.items()],
            )

Read-through caching is closely related to cache-aside but pushes the fetch-on-miss logic into the caching layer itself rather than the application code. Libraries and systems that implement a CacheLoader interface (Guava's LoadingCache in Java, or the loader pattern in some GraphQL data-loader implementations) exemplify this: the caller just asks the cache for a value, and the cache transparently loads it from the underlying source if absent, hiding the fallback logic from every call site. The practical benefit is consistency: you can't forget to populate the cache correctly in one code path while doing it right in another, because there's only one code path. Guava's LoadingCache, for instance, also handles the stampede problem discussed earlier as a first-class feature rather than something the caller has to build: concurrent calls for the same missing key block on the same in-flight load rather than triggering redundant loads, which is precisely the request-coalescing behavior implemented by hand in the Redis example above, just provided by the library instead of hand-rolled.

Choosing among these four patterns is less about which is "best" and more about matching the pattern to the read/write ratio and durability requirements of the specific data involved, and it's entirely normal, even expected, for a single system to use different patterns for different kinds of data. A typical e-commerce backend might use cache-aside for product catalog pages (read-heavy, tolerant of a short TTL), write-through for inventory counts (writes are relatively rare and correctness matters immediately), and write-behind for page-view analytics (extremely write-heavy, and losing a few seconds of counts on a crash is an acceptable trade for not needing a database write per page view). Treating "which caching strategy" as a single, system-wide decision rather than a per-data-type one is a common source of caches that are either needlessly slow on writes or needlessly risky on data that actually required stronger guarantees.

Invalidation, Consistency, and Eviction

Phil Karlton's often-quoted line that there are only two hard problems in computer science, cache invalidation and naming things, is a joke with real teeth. The core difficulty is that a cache is, by definition, a copy, and copies drift from their source unless something actively keeps them synchronized. Three broad invalidation strategies exist, and most systems combine more than one.

Time-based expiration (TTL) is the simplest and most widely used: every cached entry carries an expiration timestamp, after which it is treated as absent regardless of whether the underlying data actually changed. TTLs bound the maximum staleness a system can exhibit, which is valuable for reasoning about correctness even when it isn't perfectly tight. The trade-off is a direct tension between freshness and hit rate: a short TTL keeps data fresh but drives more traffic to the origin; a long TTL improves hit rate but risks serving stale data for longer. A detail that matters more in practice than it might seem: TTLs assigned in bulk (for example, warming a thousand keys at once with the same fixed TTL) all expire at the same instant, recreating a synchronized stampede against the origin every time that TTL boundary is crossed. The standard fix is TTL jitter, adding a small random offset (say, plus or minus ten percent) to each key's expiration so that a cohort of keys populated together doesn't also expire together.

Event-based invalidation ties cache eviction directly to the write that changed the data, either by having the write path explicitly delete or update the relevant cache key, or by publishing a change event (through something like a Kafka topic, a database change-data-capture stream via Debezium, or a simple pub/sub message) that a cache invalidation service consumes. This is more precise than TTL-based expiration because it invalidates exactly when data changes rather than on a fixed schedule, but it's also more fragile: if an invalidation event is dropped, delayed, or the code path that should trigger it is missed (a very common bug when a new write path is added and the team forgets to wire up cache invalidation), the cache silently serves stale data with no self-correcting mechanism until the TTL, if any, eventually expires. Change-data-capture-driven invalidation deserves particular attention because it decouples invalidation correctness from application code entirely: rather than trusting every write path in a codebase to remember to invalidate the right keys, a CDC pipeline reads the database's own replication or write-ahead log and emits an event for every committed change, which means invalidation coverage is structurally guaranteed rather than dependent on a developer remembering to add a cache.delete() call in a new feature. The cost is operational: running Debezium (or an equivalent) against your database's binlog or WAL is a real piece of infrastructure with its own failure modes, lag characteristics, and schema-change sensitivities.

Version-based or generation-based invalidation sidesteps explicit invalidation by embedding a version identifier into the cache key itself, so that a change to the underlying data produces a new key rather than requiring the old key to be deleted. This is the same principle as content-addressed static assets applied to arbitrary cached data: instead of asking "is this stale?", you ask "does this key still exist?" and let old, unreferenced keys expire naturally via TTL or eviction pressure. The cost is that you need a reliable way to bump the version, typically an incrementing counter or timestamp stored alongside the source data, and you accept some amount of cache bloat from orphaned old-version entries until they age out.

// Generation-based cache keys: the version is stored once per entity
// (cheap to read, since it's a single small key) and mixed into every
// derived cache key, so a single version bump invalidates every cached
// representation of that entity without deleting anything explicitly.
async function getEntityVersion(entityId: string): Promise<number> {
  const version = await redis.get(`version:${entityId}`);
  return version ? parseInt(version, 10) : 1;
}

async function bumpEntityVersion(entityId: string): Promise<void> {
  // INCR is atomic, so concurrent writers can't race and produce a
  // version that's lower than one already observed by a reader.
  await redis.incr(`version:${entityId}`);
}

async function getCachedProductCard(entityId: string): Promise<string> {
  const version = await getEntityVersion(entityId);
  const cacheKey = `product-card:${entityId}:v${version}`;

  const cached = await redis.get(cacheKey);
  if (cached) return cached;

  const rendered = await renderProductCard(entityId);
  // No TTL needed for correctness - the key is only ever "current" for
  // one version - but a TTL is still worth setting as a memory backstop.
  await redis.set(cacheKey, rendered, { EX: 86400 });
  return rendered;
}

Underneath all three strategies is a consistency model question that's easy to leave implicit: what guarantee is the cache actually making about how stale a value can be? Most application caches are eventually consistent by default, meaning a write is guaranteed to become visible everywhere only after some unspecified (or TTL-bounded) delay, which is acceptable for the large majority of web application data. Some systems need something closer to read-your-own-writes consistency instead, where a user must immediately see the effect of their own action even if other users can still see a stale view briefly; a common implementation is to bypass the cache entirely for a short window after a user's own write, either by routing that user's next few reads directly to the source of truth or by writing the fresh value into the cache synchronously as part of the write itself, effectively write-through scoped to just that one user's session. True strong consistency, where every reader everywhere sees every write immediately, is achievable but expensive, generally requiring something like a consensus protocol or a single serialized point of truth, which defeats much of the purpose of caching for read scalability in the first place; it's worth being honest with stakeholders that "always fresh" and "fast, horizontally scaled reads" are in real tension, not a solved problem you're one configuration flag away from having both of.

Eviction policy governs what happens when a cache reaches capacity and must remove entries to make room, independent of whether those entries have logically expired. Least Recently Used (LRU) evicts the entry that hasn't been accessed in the longest time, which works well when recent access predicts future access, the common case for most application workloads. Least Frequently Used (LFU) tracks access counts instead and evicts the least-popular entry, which better protects against a burst of one-time accesses evicting genuinely hot data, a known weakness of pure LRU: a single large scan or a batch job reading through a million cold keys can flush out an entire working set of genuinely hot data under strict LRU, a phenomenon sometimes called cache pollution, whereas LFU would correctly judge those million keys as low-value despite their recency.

Redis's maxmemory-policy setting exposes this choice directly, and it's worth knowing the full set of options because the right one depends on whether TTLs are set consistently across your keyspace: noeviction returns errors on writes once memory is full rather than evicting anything, appropriate when Redis is being used as more than a cache and data loss is unacceptable; allkeys-lru and allkeys-lfu apply their respective policies across the entire keyspace regardless of TTL; and volatile-lru, volatile-lfu, and volatile-ttl restrict eviction to only those keys that have a TTL set, leaving keys with no expiration untouched, which is the right choice when a single Redis instance mixes genuinely cache-like data (safe to evict) with data that functions more like a lightweight system of record (not safe to evict silently). It's also worth knowing that Redis's LRU and LFU are approximate, not exact: rather than maintaining a perfectly ordered structure of every key's access time, Redis samples a small random subset of keys and evicts the best (oldest, or least-frequently-used) candidate from that sample, which is dramatically cheaper than exact tracking and, at the sample sizes Redis uses by default, converges close enough to true LRU/LFU behavior for essentially all practical purposes.

Trade-offs and Common Pitfalls

The most consequential trade-off in any caching layer is between consistency and performance, and it is not a solvable problem so much as a dial you have to set deliberately for your domain. A social media feed can tolerate a cached view that's thirty seconds stale with no real user impact. A checkout flow showing inventory availability generally cannot, because a stale "in stock" reading translates directly into an order that fails at fulfillment. Teams get this wrong most often not by picking the wrong answer, but by never explicitly deciding, letting a cache TTL that was chosen for one use case (say, a product catalog page) get reused for a different use case (say, an inventory count) where the staleness tolerance is entirely different.

Cache stampedes, mentioned earlier, deserve emphasis as a production-incident pattern: a popular key expires, and every concurrent request that would have hit the cache instead falls through to the origin simultaneously, multiplying load on the database or backend service by however many requests were in flight. This is especially dangerous when a deploy or a cold cache start causes many keys to expire or be absent at once. Request coalescing (as shown in the TypeScript example), staggered TTLs with jitter so keys don't all expire at the same instant, and pre-warming caches before traffic shifts to them are the standard mitigations.

A second common pitfall is caching at the wrong granularity. Caching an entire API response object is simple to implement but means any single field changing invalidates the whole object, and any endpoint composing data from multiple sources can't reuse a cached fragment for the parts that didn't change. Caching at a finer granularity (per-field, or per-sub-object) improves cache reuse and reduces invalidation blast radius, but multiplies the number of cache keys and round trips needed to assemble a response, and introduces its own coordination problem if those fragments must be read consistently together.

A third, quieter pitfall is treating the cache as a database. Caches, especially in-memory ones, are volatile by design: a Redis instance without persistence enabled loses all data on restart, and even with persistence, most cache configurations are tuned for speed over durability. Systems that come to depend on the cache holding data that exists nowhere else, often starting as an innocuous "let's just store the session here for now", eventually suffer a painful incident when the cache is flushed, resized, or fails over and that data is gone. The cache should always be reconstructable from a source of truth; if it can't be, it isn't a cache anymore, it's an undocumented database.

Best Practices

A well-designed caching layer tends to share a few properties regardless of the specific technology involved. First, cache keys should be explicit and versioned, incorporating enough context (tenant ID, locale, API version, schema version) that a change in any dimension the cached value depends on naturally produces a different key rather than requiring a manual invalidation. Second, every cached value should have a bounded TTL as a backstop, even when event-based invalidation is the primary mechanism, so that a missed invalidation event degrades to "eventually correct" rather than "permanently wrong".

Observability matters as much as the caching logic itself. Hit rate, miss rate, eviction rate, and latency percentiles for both cache hits and cache misses should be tracked as first-class metrics, because a caching layer that silently degrades to a low hit rate can look identical to a healthy one from the application's perspective until load spikes and the origin can't absorb the fallback traffic. Teams that only instrument "is the cache up" and not "is the cache effective" are frequently surprised by capacity incidents that a hit-rate dashboard would have flagged days earlier.

Finally, treat cache invalidation as part of the write path's contract, not an afterthought. Code review for any change to a write path should explicitly ask what cached representations of this data exist and whether this change invalidates them, the same way a schema migration review asks about backward compatibility. Centralizing invalidation logic behind a shared function or a change-data-capture pipeline, rather than scattering cache.delete() calls across every write site, reduces the chance that a new feature quietly introduces a stale-cache bug that only surfaces in production weeks later.

Key Takeaways

Conclusion

Caching is not a single technology decision but a layered set of trade-offs repeated at every tier of a fullstack system, from the Cache-Control header a browser respects to the eviction policy a Redis instance enforces under memory pressure. The mechanisms differ across layers, but the underlying tension is constant: freshness costs latency, and latency costs freshness, and the right balance depends entirely on how much staleness your specific use case can tolerate. Engineers who treat caching as a single trick, "put Redis in front of it", tend to build systems that work well in demos and fail unpredictably under real traffic patterns, stampedes, and edge cases in invalidation.

The engineers who get the most value out of caching are the ones who reason about it explicitly: what is the source of truth, what is the acceptable staleness window, what happens when the cache is cold or unavailable, and who is responsible for invalidation when the underlying data changes. Those four questions, asked honestly at design time, resolve the vast majority of caching bugs before they reach production, and they generalize across every layer discussed here, whether you're setting an HTTP header, configuring a CDN, or writing a cache-aside function against Redis.

References