Topic 04Building Blocks
Caching
Serve repeated reads fast without hitting the database every time.
Caching is one of the highest-leverage optimizations in distributed systems. The right cache layer can reduce database load by 90%+ and drop read latency from milliseconds to microseconds.
Cache layers
Caching can happen at multiple points in your stack.
- ›CDN cache — static assets and HTML at the edge, globally distributed
- ›Application cache — in-process memory (LRU map), fastest but not shared
- ›Distributed cache — Redis or Memcached, shared across all app servers
- ›Database query cache — built into some DBs, but rarely relied on in production
Eviction policies
How the cache decides what to remove when full.
- ›LRU (Least Recently Used) — evicts what hasn't been accessed longest. Most common.
- ›LFU (Least Frequently Used) — evicts what's accessed least often.
- ›TTL (Time To Live) — data expires after a set duration regardless of access.
- ›Write-through — update cache and DB together. Consistent but slower writes.
- ›Write-back — update cache first, DB later. Fast writes, risk of data loss.
Cache invalidation
The hard part. There are only a few approaches.
- ›TTL — set expiry, accept brief staleness. Simplest.
- ›Event-driven invalidation — on DB write, explicitly delete or update cache entry.
- ›Cache-aside (lazy loading) — on cache miss, load from DB and repopulate.
- ›Write-through — always write to cache and DB together.
Cache failure modes
Know these — interviewers love them.
- ›Cache stampede — cache expires, 1000 requests hit DB simultaneously. Fix: lock or probabilistic early expiry.
- ›Cache penetration — requests for non-existent keys bypass cache every time. Fix: cache null results or bloom filters.
- ›Cache avalanche — many keys expire at once. Fix: add jitter to TTL values.
Interview tips
- ✓Estimate cache hit rate: 80/20 rule — top 20% of data handles 80% of reads
- ✓Always specify what you're caching and with what TTL
- ✓Address cache invalidation explicitly — don't leave it vague
- ✓Mention Redis by name and its data structures (strings, hashes, sorted sets)
Follow-up questions to expect
- ?How do you handle cache invalidation when data changes?
- ?What happens when your cache layer goes down?
- ?How would you size the cache? What hit rate are you targeting?
TLDR
- ›Cache what's read often and written rarely
- ›Redis for distributed cache; local LRU map for ultra-hot data
- ›LRU + TTL is the default strategy for most systems
- ›Invalidation is the hard part — pick a pattern and be consistent
- ›Know cache stampede, penetration, and avalanche — interviewers test these