Replication & Sharding
2 min read
Scaling the Database
A single database eventually can't keep up. Two techniques scale it: replication (copies) and sharding (splits).
Replication — copies for reads & safety
writes
Client ──────────▶ Primary ──replicates──▶ Replica 1 (reads)
└─────────────▶ Replica 2 (reads)
- Primary–replica: one node takes writes, replicas serve reads. Great for read-heavy workloads.
- Benefits: read scaling, high availability (promote a replica if the primary dies), backups without load.
- Cost: replication lag — replicas can be slightly stale (eventual consistency on reads).
Sharding — split data across machines
When even writes exceed one machine, partition the data. Each shard holds a subset:
| Strategy | How | Trade-off |
|---|---|---|
| Range | e.g. users A–M / N–Z | Simple, but hot spots |
| Hash | hash(key) % N |
Even spread, but resharding is painful |
| Consistent hashing | Keys on a ring | Minimal movement when adding nodes |
| Directory | Lookup table maps key → shard | Flexible, but the lookup is a bottleneck |
The pain of sharding
- Cross-shard queries and joins become expensive or impossible.
- Transactions across shards are hard (distributed transactions).
- Rebalancing when you add capacity moves data around.
- Hot shards — a celebrity user can overload one shard; pick a shard key that spreads load.
Rule of thumb: replicate before you shard. Sharding adds permanent complexity — delay it until read replicas and caching are exhausted.