Why writes are the harder half
Many designs start modest until the interviewer asks "how does it scale?" You're usually fluent on the read side — replicas, caching. The write side is often the real nightmare: bursty throughput, contention, and hard limits on a single primary.
This is the deep dive behind Common patterns. Pair with Scaling reads, Sharding, Kafka, Cassandra, and Numbers to know.
Hardware · write-optimized DBs.
Horizontal · vertical · good keys.
Queues · load shedding.
Batch · hierarchical agg.
The challenge
As the app grows from hundreds to millions of writes per second, individual servers hit walls on disk I/O, CPU, and network. Interviewers love to probe those walls — "Black Friday 4×s orders," "New Year triples drivers" — whether or not the product will ever see that traffic.
Write scaling isn't only more hardware. Architectural choices decide whether you survive runaway success or make it worse. We'll stay in a single-server design as long as we can, then go horizontal with eyes open.
Vertical scaling and database choices
First: exhaust the hardware you already have. Show due diligence — don't prematurely add Kafka and 16 shards when a bigger box or a write-friendly store would do.
Vertical scaling
Writes bottleneck on disk I/O, CPU, or network. Confirm you're hitting those walls with back-of-the-envelope math: (a) what write throughput actually is, (b) whether it fits modern hardware.
Many candidates still picture 4–8 cores and a spinning disk. Cloud and datacenter boxes often offer far more — 200 cores and 10 Gbit NICs aren't rare. Brush up on Numbers to know so you don't leave free capacity on the table.
Database choices
Next: is the store optimized for these writes? Mixed workloads are fine for most apps; write-heavy systems often win by stripping read-oriented features.
Cassandra is the classic example. Append-only commit log: sequential writes instead of in-place B-tree updates and expensive seeks. Modest hardware can do 10,000+ writes/s vs ~1,000 for a traditional RDBMS doing the same work. Trade-off: reads check multiple SSTables and merge — slower than a well-indexed Postgres. You're buying write throughput with read cost. Full dive: Cassandra.
Similar trade-offs elsewhere:
- Time-series DBs (InfluxDB, TimescaleDB) — high-volume sequential timestamped writes + delta encodings. See Time-series databases.
- Log-structured stores (LevelDB-style) — append rather than update in place.
- Column stores (ClickHouse) — batch writes efficiently for analytics.
Within any DB, write optimizations still help: disable FK constraints / heavy triggers / full-text indexes during write storms; tune WAL flushing (Postgres can batch before fsync); cut secondary indexes (faster writes, slower reads). General-purpose DBs handle mixed loads; extremes reward targeted choices.
Sharding and partitioning
Hardware exhausted — go horizontal. If one server does 1,000 writes/s, ten should do 10,000. Distribute volume so each shard handles a manageable slice. Deep dive: Sharding.
Horizontal sharding
Redis Cluster is a clean mental model: every key hashes (CRC) to a slot; slots map to nodes. Clients cache the slot map, hash the key, and talk to the owning node. Consistent hashing with virtual nodes is the other common story — see Consistent hashing.
Selecting a good partitioning key
Interviewers expect "sharding" — they want the key. A good key (hash of userId) spreads load evenly and unlocks the 10× story. A bad key (country) piles China onto one shard while New Zealand idles.
Also ask how data is read. Spread writes but force every request to hit every shard and you've built scatter-gather hell. For hottest data, arrange so common queries hit few shards. Ask: "How many shards does this request need? How often?"
- High cardinality (many distinct values)
- Even access (no celebrity hotspot unless mitigated)
- Keeps related data together for common queries
- Stable — changing keys means painful re-sharding
Vertical partitioning
Horizontal splits rows; vertical splits columns by access pattern. A social post monolith mixes write-once content, high-frequency counters, and append-only analytics — every workload interferes.
-- Before: one table hammered from all directions
TABLE posts (
id, user_id, content, media_urls, created_at,
like_count, comment_count, share_count, view_count,
last_updated
);
-- After: specialize
TABLE post_content (...); -- write-once, read-many
TABLE post_metrics (...); -- high-frequency counters
TABLE post_analytics (...); -- append-only time-series
Content → B-tree / Postgres for reads. Metrics → in-memory counters. Analytics → TSDB or column store. Data modeling is as much "how do I think about this data?" as "which machine holds it?"
Handling bursts with queues and load shedding
Sharding gets you ~80% of the way; production stumbles on bursts. Steady Amazon order volume is one thing — Black Friday 4× or New Year driver spikes are another. Autoscaling helps but isn't a panacea: spin-up takes time, and database scaling often means reduced throughput right when traffic peaks.
So either (a) buffer writes and process at a sustainable rate, or (b) drop writes the business can afford to lose.
Write queues
Kafka / SQS decouples acceptance from durable DB write. App records "accepted into queue"; clients may poll for completion. Burst absorption: DB writes at a steady rate while the queue soaks spikes. See Kafka.
Load shedding
When overwhelmed, decide which writes to keep. Strava / Uber location pings: drop a stale update and a fresher one arrives in seconds. Analytics: drop impressions, keep clicks. Release valves prevent overload from becoming total failure.
Batching and hierarchical aggregation
Sometimes you change the structure of writes. Individual ops pay network RTTs, transaction setup, index updates. Databases love batches. When the DB is the bottleneck, look upstream.
Batching
Application layer: clients or Kafka consumers batch before the DB. Fine when the app isn't source of truth (crash → re-read Kafka). Dangerous when the app confirmed the write and crashed mid-batch — data loss.
Intermediate process: a Like Batcher reads events, aggregates per post over a window, and writes one counter update. 100 likes in a minute → 1 write.
Database layer: Redis default AOF flush every 100ms batches fsyncs. Elegant but a big hammer — reserve for extremes.
Hierarchical aggregation
For analytics / live streams you often need aggregates, not every event. Live comments: millions of viewers write likes/comments; millions need the same eventually-consistent view. Naive all-to-all is intractable.
Assign viewers to broadcast nodes via consistent hashing. Write to M nodes instead of N viewers.
Root still receives every event. Same idea upstream: write processors own comment IDs, aggregate likes over a window, forward batches to the root. Aggregate up, disaggregate down — fewer writes per hop at the cost of latency.
When to use in interviews
Don't wait to be asked. Proactively name write bottlenecks, validate with napkin math, and propose deep dives.
Common scenarios
- Instagram / social — shard by userId; vertical partition (profiles vs posts vs analytics); cold storage for old posts.
- News feeds — celebrity write fan-out vs read fan-in; hybrid strategies. See Design a news feed.
- Search apps — write-heavy indexing/preprocess; partition + batch for ingest.
- Live comments — hierarchical aggregation to avoid all-to-all.
- Rate limiter / metrics / ad clicks — batching, shedding, write-optimized stores.
When NOT to use
If napkin math says one solid primary is fine, don't invent write-scaling theater. Queues buy eventual consistency and delay; partitioning can hurt reads; batching adds latency. Name trade-offs before proposing. Worst case: create a problem that wasn't there.
Common deep dives
How do you handle resharding?
8 shards → 16. Naive: take offline, rehash, move — hours of downtime. Production: gradual migration with dual-write. Write to old and new; prefer new for reads; migrate data in the background. Dual-write phase avoids loss while staying available.
What about a hot key that melts even one shard?
Even with even tweet placement, a viral post at 100K likes/s can overwhelm its shard. Two options for aggregatable metrics (likes, views, counts):
Split all keys × k: store post1Likes-0 … post1Likes-(k-1). Write load ÷ k; read = sum all. Dataset and read amplification × k. Fine if small k restores capacity.
Split hot keys dynamically: only viral keys fan out to ~100 sub-keys. Readers and writers must agree which keys are hot. Simple approach: readers always check all sub-keys; writers start splitting when local stats say hot. Announcing splits to all readers is more efficient but heavier — most systems prefer always-check.
Also related: Contention when the hot resource needs strong consistency, and Scaling reads for the mirror problem on the read path.
Cost and performance levers
Interview Q&A by level
Practice saying these out loud for scaling writes. Interviewers grade clarity and judgment more than buzzwords.
Match depth to the bar: define → trade off → operate. Don't dump principal answers in an entry-level screen.
Conclusion
Write scaling is four strategies working together: vertical + database choice, sharding and partitioning, queues and load shedding, batching and hierarchical reducers. Successful interviews don't overcomplicate — they find where each is required and apply it surgically.
Easy mistake: employ write scaling when napkin math says no. Sharding is the expected starting move once vertical is exhausted. High-volume analytics/numeric data → batching and hierarchy for immediate 5–10×. Queues and shedding when requirements allow delay or drops.
Reduce throughput per component — whether you spread 10K writes across 10 shards, smooth bursts through a queue, or batch into 100 bulk ops.
Related: Common patterns · Scaling reads · Sharding · Scalability.