SD notes

Caching

Beginner · 7 min read · updated 29 October 2026

Quick answer

What is caching in system design?

Caching keeps copies of frequently read data in a fast store, usually memory, so most reads skip the slow source such as a database. It cuts latency and load, but adds the work of keeping copies fresh: choosing a read and write policy, an eviction policy such as LRU, and a TTL that bounds how stale a copy can be.

On this page

Where caches live
Cache-aside (lazy loading)
Eviction: LRU
Write policies
TTL and staleness
Hit ratio and pitfalls
Interview questions
Common questions

A cache keeps hot data close and fast, so most reads never reach the slow store behind it.

Where caches live

Client: browser cache, HTTP Cache-Control headers.
Edge: a CDN for static files and cacheable responses.
Application: an in-process map (fastest, but per server).
Distributed: Redis or Memcached shared by every app server.
Database: its own buffer pool and query caches.

Cache-aside (lazy loading)

Read: cache → on a miss, database → fill the cache → return.
Write: update the database, then delete the cached key (the next read refills it).
Only what is read gets cached; a cache outage means slower, not broken.
The most common pattern for read-heavy services.

Cache-aside: check the cache, fall back to the database, fill the cache

Read 1 asks for A. The app checks the cache first, but A is not there: a miss.

0:01 / 0:25Step 1 of 10

Transcript · 10 steps

Eviction: LRU

Memory is finite; when full, something must go.
LRU: evict the least recently used — recent use predicts reuse.
O(1) with a hash map + doubly linked list.
Others: LFU (least frequently used), FIFO, random; Redis offers approximate LRU and LFU.

LRU eviction on a capacity-3 cache, access by access

An LRU cache holds 3 items, kept in order of last use. When it is full, a miss evicts the least recently used item.

0:01 / 0:25Step 1 of 10

Transcript · 10 steps

Write policies

	Write-through	Write-back	Write-around
Write goes to	Cache and database	Cache only, flushed later	Database only
Write speed	Slower	Fastest	Database speed
Risk	None extra	Loses unflushed writes	First read after a write misses
Good for	Read-after-write data	Write-heavy, loss-tolerant	Data rarely re-read

Write-through, write-back and write-around: one write, then one read

Every lane starts with x = 3 in both the cache and the database. Each app now writes x = 5.

0:01 / 0:22Step 1 of 7

Transcript · 7 steps

TTL and staleness

A TTL expires an entry after a fixed time, so a stale copy cannot live forever.
Short TTL: fresher data, more misses. Long TTL: more hits, staler data.
TTL bounds staleness; delete-on-write removes most of it.
Add random jitter to TTLs so popular keys do not all expire together.

TTL expiry: a cached value lives 5 seconds, stale or not

At 0s the cache is empty: a miss. The value v1 is read from the database and cached with a 5-second TTL, so it expires at 5s.

0:01 / 0:16Step 1 of 6

Transcript · 6 steps

Hit ratio and pitfalls

Hit ratio = hits ÷ lookups; average read ≈ hit × cache time + miss × (cache + database time).
Stampede: a hot key expires and every request misses at once — refill once, others wait.
Penetration: requests for keys that never exist always miss — cache the "not found".
Hot key: one key overloads one cache node — replicate it or keep a local copy.
Size the cache for the working set, not the whole dataset.

Interview questions

What would you cache? Data read far more often than written and tolerant of brief staleness: profiles, product pages, computed feeds.
Why delete the key on write instead of updating it? Two concurrent updates can write the cache in the wrong order; a delete just forces the next read to fetch the truth.
Redis or Memcached? Memcached is a simple multithreaded key-value cache; Redis adds data structures, persistence, replication and pub/sub.
How do you scale a cache across nodes? Partition keys with consistent hashing, so adding a node moves only a fraction of keys.

Next: Content Delivery Networks · Before this: Load Balancing

Common questions

What is the cache-aside pattern?

In cache-aside (lazy loading) the application checks the cache first. On a hit it returns the cached value; on a miss it reads the database, stores the result in the cache and returns it. Only data that is actually read gets cached, and a cache failure just means slower reads.

How does an LRU cache work?

An LRU (least recently used) cache keeps entries in order of last access. A hit moves the entry to the most-recent end; a miss on a full cache evicts the entry at the least-recent end. A hash map plus a doubly linked list gives O(1) get and put.

What is the difference between write-through and write-back?

Write-through writes to the cache and the database before acknowledging, so both always agree but writes are slower. Write-back writes only the cache and acknowledges at once, flushing to the database later; writes are fast, but data not yet flushed is lost if the cache node fails.

What is a cache hit ratio?

The hit ratio is hits divided by total lookups. It decides how much a cache helps: average read time is roughly hit ratio × cache time + miss ratio × (cache time + database time), so moving from a 90% to a 99% hit ratio cuts database reads tenfold.

What is cache invalidation and why is it hard?

Invalidation is removing or updating a cached copy when the source changes. It is hard because the cache and the database are separate systems: a write and a concurrent read can race and leave an old value cached. TTLs, deleting on write and versioned keys limit the damage.

What is a cache stampede?

A stampede (thundering herd) happens when a popular key expires and many requests miss at once, all hitting the database together. Fixes include letting one request refill while others wait (request coalescing), refreshing before expiry, and adding random jitter to TTLs.

Caching in System Design: Cache-Aside, LRU and TTL

Caching for system design interviews: cache-aside, LRU eviction, write-through vs write-back vs write-around, TTL and hit ratio, each shown as an animation.

Subject: System Design
Level: Beginner
Reading time: 7 min
Updated: 2026-10-29

What is caching in system design?

A cache keeps hot data close and fast, so most reads never reach the slow store behind it.

Where caches live

Client: browser cache, HTTP Cache-Control headers.
Edge: a CDN for static files and cacheable responses.
Application: an in-process map (fastest, but per server).
Distributed: Redis or Memcached shared by every app server.
Database: its own buffer pool and query caches.

Cache-aside (lazy loading)

Read: cache → on a miss, database → fill the cache → return.
Write: update the database, then delete the cached key (the next read refills it).
Only what is read gets cached; a cache outage means slower, not broken.
The most common pattern for read-heavy services.

Cache-aside: check the cache, fall back to the database, fill the cache.

Read 1 asks for A. The app checks the cache first, but A is not there: a miss.
On a miss the app itself reads A from the database — the slow path. The cache never talks to the database in cache-aside.
The app stores A in the cache, then returns it. Miss 1; the next read of A will be a hit.
Read 2 asks for B. The app checks the cache first, but B is not there: a miss.
On a miss the app itself reads B from the database — the slow path. The cache never talks to the database in cache-aside.
The app stores B in the cache, then returns it. Miss 2; the next read of B will be a hit.
Read 3 asks for A. The app checks the cache first, and A is there: a hit, with no database work.
The cached A goes straight back to the app — 1 hit so far, the fast path that makes caching worth it.
Read 4 asks for B. The app checks the cache first, and B is there: a hit, with no database work.
The cached B goes straight back to the app — 2 hits so far, the fast path that makes caching worth it.

Eviction: LRU

Memory is finite; when full, something must go.
LRU: evict the least recently used — recent use predicts reuse.
O(1) with a hash map + doubly linked list.
Others: LFU (least frequently used), FIFO, random; Redis offers approximate LRU and LFU.

LRU eviction on a capacity-3 cache, access by access.

An LRU cache holds 3 items, kept in order of last use. When it is full, a miss evicts the least recently used item.
Access A: a miss. There is still room, so A is loaded and goes in at the most-recent end.
Access B: a miss. There is still room, so B is loaded and goes in at the most-recent end.
Access C: a miss. There is still room, so C is loaded and goes in at the most-recent end.
Access A: a hit. A moves to the most-recent end, giving A, C, B; nothing is evicted.
Access B: a hit. B moves to the most-recent end, giving B, A, C; nothing is evicted.
Access D: a miss with the cache full, so C, the least recently used, is evicted to make room: D, B, A.
Access A: a hit. A moves to the most-recent end, giving A, D, B; nothing is evicted.
Access E: a miss with the cache full, so B, the least recently used, is evicted to make room: E, A, D.
Access A: a hit. A moves to the most-recent end, giving A, E, D; nothing is evicted. In all, 4 hits out of 9 accesses: a hit ratio of 44%.

Write policies

	Write-through	Write-back	Write-around
Write goes to	Cache and database	Cache only, flushed later	Database only
Write speed	Slower	Fastest	Database speed
Risk	None extra	Loses unflushed writes	First read after a write misses
Good for	Read-after-write data	Write-heavy, loss-tolerant	Data rarely re-read

Write-through, write-back and write-around: one write, then one read.

Every lane starts with x = 3 in both the cache and the database. Each app now writes x = 5.
Write-through and write-back update the cache first. Write-around skips the cache, writes the database and drops the cached copy.
Write-back acknowledges after 1 hop, leaving the database at x = 3 — dirty data a cache crash would lose. Write-around is acknowledged too; write-through is still writing the database.
Write-through acknowledges only after both copies hold x = 5: 2 hops, slower writes, but cache and database always agree.
Now each app reads x. Write-through and write-back hit the cache; write-around misses, because its write bypassed the cache.
Write-around fetches x = 5 from the database and fills the cache. Write-back's background flush finally writes x = 5 to the database.
Settled: every lane holds x = 5 in both places. The trade-off was write speed (2 vs 1 hops to acknowledge) against durability and read-after-write hits.

TTL and staleness

A TTL expires an entry after a fixed time, so a stale copy cannot live forever.
Short TTL: fresher data, more misses. Long TTL: more hits, staler data.
TTL bounds staleness; delete-on-write removes most of it.
Add random jitter to TTLs so popular keys do not all expire together.

TTL expiry: a cached value lives 5 seconds, stale or not.

At 0s the cache is empty: a miss. The value v1 is read from the database and cached with a 5-second TTL, so it expires at 5s.
At 2s the entry is 2s old, younger than its 5s TTL: a hit on v1, matching the database.
The database changed to v2 at 3s, but at 4s the cache still serves v1 until it expires. A TTL bounds staleness; it does not prevent it.
At 6s the entry has expired (at 5s), so the read misses and refills the cache with v2, valid until 11s.
At 8s the entry is 2s old, younger than its 5s TTL: a hit on v2, matching the database.
At 10s: a hit. Over 6 reads, 4 hits (67%), 1 of them stale. A shorter TTL means fresher data but more misses.

Hit ratio and pitfalls

Hit ratio = hits ÷ lookups; average read ≈ hit × cache time + miss × (cache + database time).
Stampede: a hot key expires and every request misses at once — refill once, others wait.
Penetration: requests for keys that never exist always miss — cache the "not found".
Hot key: one key overloads one cache node — replicate it or keep a local copy.
Size the cache for the working set, not the whole dataset.

Interview questions

What would you cache? Data read far more often than written and tolerant of brief staleness: profiles, product pages, computed feeds.
Why delete the key on write instead of updating it? Two concurrent updates can write the cache in the wrong order; a delete just forces the next read to fetch the truth.
Redis or Memcached? Memcached is a simple multithreaded key-value cache; Redis adds data structures, persistence, replication and pub/sub.
How do you scale a cache across nodes? Partition keys with consistent hashing, so adding a node moves only a fraction of keys.