Medium · Mobility and logistics
Fleet telemetry
A freight company has 150,000 trucks that each send a batch of sensor readings every 30 seconds. Dispatchers watch live maps and analysts run reports. You must take in every batch, keep a month of detail cheaply, and survive the moments when thousands of trucks come back online at once. Practise this medium system design problem for free with a Google or GitHub account.
The brief
A freight company has 150,000 trucks that each send a batch of sensor readings every 30 seconds. Dispatchers watch live maps and analysts run reports. You must take in every batch, keep a month of detail cheaply, and survive the moments when thousands of trucks come back online at once.
Think of trucks reporting every few seconds.
Users, scale and requirements
- Who uses it
- 150,000 trucks and 3,000 dispatchers and analysts (approx.)
- Audience in plain numbers
- 3,000 users or devices
- Normal peak
- ≈ 6,000 requests per second
- Request mix
- Telemetry batch: 93%; Report: 3%; Read: 4%
- Speed for 99 of 100 requests
- Report: ≈ 5,000 ms; Read: ≈ 300 ms
- Successful requests
- 99.9%
- Data safety
- At most 1 minute of writes may be lost.
- Monthly budget
- ≈ $9,700
- Data at rest
- ≈ 2 TB
- Where users are
- North America: 100%
- Home region
- North America
- Waiting deadlines
- Telemetry batch: 3,600 seconds; Report: 3,600 seconds; Read: 900 seconds
- Minimum retained history
- 30 days on authoritative stores
≈ marks an approximation, just as in the game. Waiting deadlines measure the age of queued work and are separate from interactive response time.
- Each telemetry batch holds about 100 readings, about 1 KB. 150,000 trucks every 30 s is about 5,000 batches a second.
- About 500,000 readings a second at 1.5 bytes each compressed is around 65 GB a day (approx.).
- All numbers are rough approximations.
- Scored data retention: at least 30 days on active authoritative stores. Copies used only for search or reports can be rebuilt. Local device history and rollup policies still require the separate protocol check.
Questions that shape the design
Can a batch from a truck be lost?
A minute of readings may be lost in a disaster, but not routinely. Batches are used for safety and billing reports.
How old can the live map data be?
A few seconds is fine. Reports can take several seconds to run.
How long do we keep the readings?
30 days of full detail on fast storage, then the older data moves somewhere cheaper.
What do trucks do when they lose signal?
They keep the readings and upload the backlog when the signal returns, so reconnects arrive in bursts.
The traffic day
Traffic concentrated in working hours. The clock covers 00:00 to 24:00 in the home region. Events add pressure on top of that normal traffic.
07:00 · Synchronized rush
Many callers send requests together instead of spreading them over time.
Traffic for Telemetry batch reaches 4 times the scheduled load; rises over 10 seconds, holds for 60 seconds and eases over 300 seconds. Other request types keep their scheduled load.
11:00 · Metrics explosion
A metric label creates many more distinct series, increasing the bill.
Billed usage multiplier: 20. Duration: ≈ 21,600 seconds.
14:30 · Disk full
Data or logs fill the disk. Writes stop until someone frees space.
Lasts until detected and repaired.
What a strong design has to handle
- Trucks do not wait for an answer. What can accept a batch immediately and let the rest of the system work at its own pace?
- Ingest reconnect batches once and separate event time from arrival time.
- A stream costs a fixed amount each month. A per-message queue looks cheaper to start and costs more at this volume.
- Short stream retention (one day) saves money but means a stuck worker must be fixed within a day or data is lost.
Make a prediction, run a full traffic day, then compare the result. The in-game editorial opens after a finished day and includes measured reference designs.
Concepts to practise
Related problems
- Medium
Factory sensors
A factory has thousands of machines, each sending readings many times a second. Engineers open charts of the last hour and last month, and an alarm must fire within seconds when a reading goes out of range. The data grows without end.
- Hard
Email service
Mail arrives from the whole internet, including a lot of spam, and must land in the right inbox. People open, search, send and attach. Never lose a message, and keep search fast even over years of mail.
- Hard
Object storage service
Customers store billions of files, from tiny pictures to huge videos, and fetch them by name. The service must never lose a file, scale without a limit and stay cheap. Most files are written once and rarely read again.
- Medium
Smart power grid
A power company has 5 million smart meters. Each reports usage every 15 minutes, and they all try on the quarter hour. Customers see their usage, billing needs exact totals for seven years, and a regulator checks that records are kept safely. Control commands to home batteries and thermostats must never run twice.