Hard · Social and messaging
Group chat and presence
Teams talk in channels with many people. Everyone sees messages in the same order, sees who is online and who is typing, and gets notified when mentioned. At nine every morning thousands of workplaces start at once, and one announcement can reach a channel of tens of thousands. Practise this hard system design problem for free with a Google or GitHub account.
The brief
Teams talk in channels with many people. Everyone sees messages in the same order, sees who is online and who is typing, and gets notified when mentioned. At nine every morning thousands of workplaces start at once, and one announcement can reach a channel of tens of thousands.
Think of team chat like Slack or Discord.
Users, scale and requirements
- Who uses it
- 3 million people a month
- Audience in plain numbers
- 3,000,000 people per month
- Normal peak
- ≈ 25,000 requests per second
- Request mix
- Message: 45%; Analytics event: 20%; Read: 15%; Write: 5%; Sync change: 5%; Notification: 5%; Search: 3%; Sign-in: 2%
- Speed for 99 of 100 requests
- Message: ≈ 300 ms; Read: ≈ 400 ms; Write: ≈ 400 ms; Sync change: ≈ 400 ms; Search: ≈ 700 ms; Sign-in: ≈ 1,000 ms
- Successful requests
- 99.9%
- Data safety
- No acknowledged write may be lost.
- Monthly budget
- ≈ $74,000
- Data at rest
- ≈ 25 TB
- Where users are
- North America: 50%; Europe: 35%; Asia Pacific: 15%
- Home region
- North America
- Waiting deadlines
- Message: 900 seconds; Analytics event: 3,600 seconds; Read: 900 seconds; Write: 3,600 seconds; Sync change: 900 seconds; Notification: 900 seconds; Search: 900 seconds; Sign-in: 900 seconds
- Fresh reads
- People must see their own changes. At most 0.1% of reads may be stale in the model.
- Personal reads
- 90% of reads are private; a shared public copy cannot answer them.
- Required proofs
- Audit trail; Managed encryption keys
≈ marks an approximation, just as in the game. Waiting deadlines measure the age of queued work and are separate from interactive response time.
- Presence updates ("online", "typing") are the biggest source of events (approx.).
- A message sent in a 50,000-person channel becomes up to 50,000 deliveries.
- Notification speed measures provider acceptance. Device receipt is outside this traffic model; check expiry, provider outcome and retries in the correctness trace before claiming delivery guarantees.
Questions that shape the design
Must everyone see messages in the same order?
Yes. A conversation that reads differently for two people is broken.
Is "online" status worth saving forever?
No. It is a guess that expires in half a minute. It is gone if nobody refreshes it.
How big is the biggest channel?
A company-wide channel might have 50,000 members. A message there reaches everyone online.
What do businesses ask for?
A record of who did what, and proof that their messages are encrypted with keys managed in a key service.
The traffic day
Traffic concentrated in working hours. The clock covers 00:00 to 24:00 in the home region. Events add pressure on top of that normal traffic.
09:00 · Synchronized rush
Many callers send requests together instead of spreading them over time.
Traffic for Sign-in, Message, Read, Sync change reaches 10 times the scheduled load; rises over 10 seconds, holds for 60 seconds and eases over 300 seconds. Other request types keep their scheduled load.
14:00 · Celebrity post
A popular account posts, multiplying the work needed to update follower feeds.
Work multiplier: 20. Duration: ≈ 900 seconds.
16:00 · Network partition
Your regions cannot talk to each other during the partition.
Duration: ≈ 300 seconds.
What a strong design has to handle
- Presence changes constantly and is worthless after a minute. Should it live in the same database as messages?
- Agree on channel message order while treating presence as expiring soft state.
- One writer per channel keeps order simple, but a very busy channel is then limited by that writer. Splitting a channel breaks the order guarantee.
- Presence in a cache with an expiry is cheap and self-cleaning, but it can be wrong for up to half a minute after a phone vanishes.
Make a prediction, run a full traffic day, then compare the result. The in-game editorial opens after a finished day and includes measured reference designs.
Concepts to practise
Related problems
- Hard
Live streaming
Streamers broadcast live and thousands watch together, with a chat on the side. Everyone wants the same piece of video at the same second, so nothing can be saved for later. When a big streamer goes live, the whole audience arrives at once.
- Hard
Collaborative documents
Many people edit the same document at the same moment and everyone must see a consistent result in a fraction of a second. Edits must never be lost, even if a server crashes in the middle of typing or two people type in the same place.
- Medium
One-to-one chat
People send short messages to each other from their phones. A message must arrive once, in order, even when the phone is on a bad connection and sends it again. If the other person is offline, the message waits and a notification goes out.
- Hard
Social feed
Every person opens the app and sees a feed of posts from the people they follow. Most people have a few hundred followers, but a few have millions, and when they post, millions of feeds must change. The feed must be fast and your own post must show up at once.