Design a News Feed

Design a News Feed (Twitter / Instagram)

A ranked, personalised stream of posts from accounts a user follows. The central tension is fan-out.

1. Requirements

• Functional: publish a post; view a feed of followed accounts, newest/ranked first.
• Non-functional: read-heavy, low feed latency (< 200 ms), eventual consistency is fine (a post can appear a few seconds late).

2. The core question: when do we build the feed?

Fan-out on write (push) — when you post, copy it into every follower's precomputed feed.

• ✅ Feed reads are instant (just read your list).
• ❌ A celebrity with 50M followers triggers 50M writes — the fan-out problem.

Fan-out on read (pull) — build the feed on demand by querying everyone you follow.

• ✅ Cheap writes, no wasted work for inactive users.
• ❌ Slow reads; heavy DB load per feed request.

3. The hybrid (what real systems do)

• Push for normal users (precompute feeds into a cache like Redis).
• Pull for celebrities — fetch their recent posts at read time and merge them into the precomputed feed.

This caps write amplification while keeping most reads fast.

4. High-level design

Post ─▶ Post service ─▶ DB
                     └─▶ Fan-out service ─▶ Redis feed lists (per user)

Feed ─▶ Feed service ─▶ Redis (precomputed) + pull celebrity posts ─▶ Rank ─▶ Client

5. Data model

posts:    id | author_id | content | media_url | created_at
follows:  follower_id | followee_id
feeds:    user_id → [post_id, post_id, ...]   (in Redis, capped to ~1000)

6. Deep dives

• Ranking: newest-first is easy; engagement ranking needs a scoring model (recency + affinity + predicted engagement).
• Storage: keep feeds bounded (last N posts) — nobody scrolls 10,000 items.
• Media: store in object storage (S3) + CDN; the feed holds only URLs.
• Pagination: use cursor-based (since_id) not offset — stable under inserts.

Takeaway: news feeds are a fan-out problem. The hybrid push/pull model is the standard answer.