Medium · IoT and devices
Factory sensors
A factory has thousands of machines, each sending readings many times a second. Engineers open charts of the last hour and last month, and an alarm must fire within seconds when a reading goes out of range. The data grows without end. Practise this medium system design problem for free with a Google or GitHub account.
The brief
A factory has thousands of machines, each sending readings many times a second. Engineers open charts of the last hour and last month, and an alarm must fire within seconds when a reading goes out of range. The data grows without end.
Think of sensors on a factory floor.
Users, scale and requirements
- Who uses it
- 10,000 sensors and 300 engineers (approx.)
- Audience in plain numbers
- 300 users or devices
- Normal peak
- ≈ 3,000 requests per second
- Request mix
- Telemetry batch: 90%; Read: 6%; Report: 2%; Analytics event: 2%
- Speed for 99 of 100 requests
- Read: ≈ 800 ms; Report: ≈ 10,000 ms
- Successful requests
- 99.9%
- Data safety
- At most 1 minute of writes may be lost.
- Monthly budget
- ≈ $9,000
- Data at rest
- ≈ 20 TB
- Where users are
- North America: 100%
- Home region
- North America
- Waiting deadlines
- Telemetry batch: 3,600 seconds; Read: 900 seconds; Report: 3,600 seconds; Analytics event: 5 seconds
- Minimum retained history
- 5 days on authoritative stores
≈ marks an approximation, just as in the game. Waiting deadlines measure the age of queued work and are separate from interactive response time.
- Machines send readings throughout the day, without a quiet night.
- Each reading is about 100 bytes (approx.).
- All numbers are rough approximations.
- Scored data retention: at least 5 days on active authoritative stores. Copies used only for search or reports can be rebuilt. Local device history and rollup policies still require the separate protocol check.
Questions that shape the design
How quickly must an alarm fire?
Within 5 seconds of the bad reading.
How long do we keep full detail?
Five days in full. After that, one average per minute is enough for a year.
Can the factory network drop?
Yes. Readings wait on the machine and arrive later; late data must still be stored.
The traffic day
Fairly steady traffic throughout the day. The clock covers 00:00 to 24:00 in the home region. Events add pressure on top of that normal traffic.
08:00 · Synchronized rush
Many callers send requests together instead of spreading them over time.
Traffic for Telemetry batch reaches 3 times the scheduled load; rises over 10 seconds, holds for 60 seconds and eases over 300 seconds. Other request types keep their scheduled load.
15:00 · Network partition
Your regions cannot talk to each other during the partition.
Duration: ≈ 300 seconds.
What a strong design has to handle
- Data arrives non-stop and never goes away. What do you keep in full and what do you shrink?
- Detect timely alarms from event-time data without letting history overwhelm ingestion.
- Rolling up to one-minute averages cuts old data about 600 times, but you lose the short spikes.
- A stream costs extra, but it lets alarms and storage work at their own pace and replay after a failure.
Make a prediction, run a full traffic day, then compare the result. The in-game editorial opens after a finished day and includes measured reference designs.
Concepts to practise
Related problems
- Medium
Fleet telemetry
A freight company has 150,000 trucks that each send a batch of sensor readings every 30 seconds. Dispatchers watch live maps and analysts run reports. You must take in every batch, keep a month of detail cheaply, and survive the moments when thousands of trucks come back online at once.
- Hard
Object storage service
Customers store billions of files, from tiny pictures to huge videos, and fetch them by name. The service must never lose a file, scale without a limit and stay cheap. Most files are written once and rarely read again.
- Hard
Health readings from a wearable
A wearable records health readings even when its phone or cloud service is unavailable. Keep raw readings on the device when possible and upload only consented summaries in sparse batches. The system must not lose locally saved readings during a supported outage, and deleting an account must have a verifiable end-to-end workflow.
- Hard
Email service
Mail arrives from the whole internet, including a lot of spam, and must land in the right inbox. People open, search, send and attach. Never lose a message, and keep search fast even over years of mail.