Week 2 — Load Balancers & Caching
Load Balancers
Section titled “Load Balancers”A load balancer distributes traffic across multiple servers. Without it, one server becomes the bottleneck and a single point of failure.
L4 vs L7
Section titled “L4 vs L7”| Layer 4 (TCP/UDP) | Layer 7 (HTTP) | |
|---|---|---|
| Sees | IP + port | Full HTTP request |
| Routing | By IP/port | By path, header, cookie |
| Performance | Faster | Slower but smarter |
| Example | AWS NLB | Nginx, HAProxy, AWS ALB |
Algorithms
Section titled “Algorithms”| Algorithm | How | Best for |
|---|---|---|
| Round Robin | 1→2→3→1→2→3 | Uniform request weight |
| Weighted Round Robin | Server A gets 3x traffic of B | Mixed server capacity |
| Least Connections | Route to server with fewest active sessions | Long-running requests (e.g. Odoo) |
| IP Hash | Hash(client IP) → same server always | Session stickiness |
Caching
Section titled “Caching”Cache layers (closest to user first):
Browser cache → CDN → App cache (Redis) → DB query cache → DB buffer poolEviction Policies
Section titled “Eviction Policies”- LRU (Least Recently Used) — evict the least recently accessed item. Most common.
- LFU (Least Frequently Used) — evict the least accessed over time.
- TTL — expire after a fixed duration regardless of access.
In practice: LRU + TTL together.
Cache Strategies
Section titled “Cache Strategies”| Strategy | Write | Read | Stale risk | Complexity |
|---|---|---|---|---|
| Cache-aside | App writes DB, invalidates cache | App reads cache, misses → DB | Medium | Low |
| Write-through | App writes cache + DB together | Always fresh | Low | Medium |
| Write-behind | App writes cache, async flush to DB | Fresh | Low (risk: data loss) | High |
| Read-through | Cache populates itself on miss | Auto-populated | Medium | Medium |
Most practical: cache-aside. Explicit control, easy to reason about.
Cache Stampede
Section titled “Cache Stampede”When cache expires and 1000 concurrent requests all miss and hit the DB simultaneously.
Solutions:
- Mutex lock — only one request fetches, others wait
- Background refresh — refresh before expiry, never let it expire cold
- Probabilistic early expiry — randomly refresh slightly before TTL
Practice
Section titled “Practice”Design caching for a product catalogue:
- 100k products, updated rarely
- 50k concurrent users browsing
- What do you cache? For how long? What invalidates it?
Self-check
Section titled “Self-check”- When does Least Connections beat Round Robin?
- Why is cache invalidation “one of the two hardest problems in computer science”?
- What’s the difference between a CDN cache and a Redis cache?