Why caching shows up every time
In system design interviews, caching appears whenever you need to handle high read traffic. The database becomes the bottleneck, latency climbs, and the interviewer waits for you to say cache.
A user profile from Postgres might take ~50ms; the same key from Redis is often ~1ms. Disk vs memory. Caches cut load and latency — and create new problems around invalidation and failure.
CDN · Redis · client · in-process.
Aside · through · behind · read-through.
LRU default · TTL for freshness.
Stampede · staleness · hot keys.
Where to cache
Most engineers hear “cache” and picture Redis between the app and the database. That’s the interview default — but caching also lives in browsers, CDNs, process memory, and inside the DB itself.
External caching
A standalone cache your app talks to over the network — Redis or Memcached. Shared across app servers, with LRU (or similar) and TTL so memory stays bounded.
CDN
Edge servers cache content close to users. Modern CDNs can cache static files, public HTML, and even API responses — but in interviews, introduce a CDN first for static media at scale. Expand only if the problem needs more.
- User requests an image.
- Nearest edge serves it on hit.
- On miss, edge fetches origin, stores a copy, returns it.
- Later users in that region hit the edge.
Client-side caching
Store data close to the requester: browser HTTP cache / localStorage, mobile on-device storage, or even client libraries (Redis clients cache cluster slot maps so they route without re-querying the cluster every op). You have limited backend control — data goes stale and global invalidation is hard.
In-process caching
Use the app’s own RAM for tiny, hot values: config, feature flags, reference data, rate-limit counters, precomputed constants. Faster than Redis (no network). Each instance has its own copy — updates on one server don’t notify the others.
Cache architectures
How you read and write the cache changes performance, consistency, and complexity. Four patterns cover almost every interview.
Cache-aside (lazy loading)
The default. App checks cache → hit returns → miss loads DB, populates cache, returns. Only caches what’s needed; a miss costs an extra trip.
Write-through
App writes to the cache; the cache (or library) synchronously writes the DB before acknowledging. Fresher reads after writes; slower writes. Redis doesn’t do this natively — you need app code or a framework. Dual-write failures still need retries; perfect consistency without distributed transactions is hard.
Write-behind (write-back)
App writes only to the cache; the cache batches async flushes to the DB. Very fast writes; if the cache dies before flush, you can lose data. Fit for analytics / metrics where occasional loss is acceptable — not payments.
Read-through
Cache is a smart proxy: app never talks to the DB. On miss, the cache fetches, stores, and returns. Read twin of write-through. CDNs work this way. For Redis application caches, cache-aside is far more common — don’t force read-through unless you’re discussing CDN-like infrastructure.
Cache eviction policies
Memory is finite. Eviction decides what dies when the cache is full. TTL bounds freshness — often paired with an eviction policy, not a substitute for one.
- LRU — drop least recently used. Adapts to most workloads; interview default.
- LFU — drop least frequently used. Good when popularity is stable (catalogs, evergreen content). Approximate frequency with Count-Min Sketch only when exact counters won't fit — see Specialized data structures.
- FIFO — drop oldest insert. Ignores hotness; rarely preferred.
- TTL — expire by age. Essential for sessions, API responses, anything that must refresh.
Common caching problems
If you propose a cache, show you can handle the failure modes — not only the speedup.
Cache stampede (thundering herd)
A popular key expires and hundreds or thousands of requests miss at once, all rebuilding from the DB. Homepage feed with a 60s TTL: at expiry, peak traffic can spike the primary.
- Request coalescing / singleflight — one rebuild; others wait. Most effective fix.
- Cache warming / early refresh — rebuild before hard expiry (helps TTL expiry; less useful if you only invalidate on write).
- Probabilistic early expiration — stagger rebuilds so they don’t align.
Cache consistency
Cache and DB disagree. Common when you read from cache but write the DB first — a window of stale reads. No perfect fix; pick by how fresh the product must be.
- Invalidate on write — delete the key after a successful DB update; next read refills.
- Short TTL — bound staleness when eventual consistency is OK.
- Accept lag — feeds, metrics, analytics often tolerate seconds of delay.
Hot keys
One key gets disproportionate traffic. Hit rate looks healthy while a single Redis shard melts — e.g. a celebrity profile under viral load.
Caching in the interview
Bring up caching when you’ve named a problem: read-heavy load, expensive joins, high DB CPU on repeated queries, or a sub-10ms latency bar that disk can’t meet. Rough numbers help — see Numbers to know.
- Identify the bottleneck — “Profile reads at 500/s, 30ms each.”
- Decide what to cache — frequent reads, infrequent writes, expensive to compute. Name keys:
user:123:profile,trending:posts:global. - Choose architecture — cache-aside by default; CDN for media; in-process for extreme hot keys.
- Eviction + TTL — LRU + TTL tied to how wrong the UX can look; invalidate on write when freshness matters sooner.
- Name one downside — stampede, Redis outage → DB crush (circuit breaker / fallback), or lost invalidates.
- Junior — “TTL of 5 minutes for everything.” Senior — TTL matches blast radius.
- Junior — “Always update the cache on write.” Senior — often invalidate; dual-write has its own bugs.
- Junior — “Miss rate doesn’t matter if Redis is fast.” Senior — 50% miss at 10k QPS still dumps 5k QPS on the DB.
Cost and performance levers
Cache decision tree
Invalidation strategies
- TTL only — simple, stale window.
- Write-time invalidate — delete key on update.
- Versioned keys —
user:123:v7bump version. - Event bus — publish invalidation to all app nodes.
Failure modes to mention
Call out at least one dependency failure (DB down, cache stampede, queue lag, region outage) and your mitigation (timeouts, retries with jitter, degraded mode, circuit breaker).
Interview Q&A by level
Practice saying these out loud for caching. Interviewers grade clarity and judgment more than buzzwords.
Match depth to the bar: define → trade off → operate. Don't dump principal answers in an entry-level screen.
Wrapping up
Caching is what you do when reading the database is too slow or too expensive. The trade-off is simple: faster reads and less load behind the cache, in exchange for staleness, invalidation complexity, and new failure modes.
- Identify the bottleneck, then propose the cache.
- Default: Redis + cache-aside + TTL + invalidate-on-write.
- CDN for static media; in-process for tiny ultra-hot keys.
- Don’t cache everything — a well-indexed DB is enough more often than candidates admit.
Pair with indexing (fix the query first), Scaling reads (full ladder: index → replicas → cache → CDN), scalability, and Redis for a deep dive on the default cache.