Easy · Finance and payments
Payroll run
On pay day your system tells the payment rail to move salaries for two million employees, then publishes a payslip for each one. Employees open the app at breakfast to see if they were paid. Paying anyone twice is the one thing you must never do. Practise this easy system design problem for free with a Google or GitHub account.
The brief
On pay day your system tells the payment rail to move salaries for two million employees, then publishes a payslip for each one. Employees open the app at breakfast to see if they were paid. Paying anyone twice is the one thing you must never do.
Think of a payroll service like Gusto.
Users, scale and requirements
- Who uses it
- 2 million employees at 40,000 companies
- Audience in plain numbers
- 2,000,000 users or devices
- Normal peak
- ≈ 2,400 requests per second
- Request mix
- Read: 30%; Sign-in: 10%; Write: 5%; Batch task: 10%; Payment: 30%; Report: 15%
- Speed for 99 of 100 requests
- Read: ≈ 500 ms; Sign-in: ≈ 1,000 ms; Write: ≈ 500 ms; Payment: ≈ 3,000 ms; Report: ≈ 4,000 ms
- Successful requests
- 99.5%
- Data safety
- No acknowledged write may be lost.
- Monthly budget
- ≈ $8,600
- Data at rest
- ≈ 2 TB
- Where users are
- North America: 100%
- Home region
- North America
- Waiting deadlines
- Read: 900 seconds; Sign-in: 900 seconds; Write: 3,600 seconds; Batch task: 3,600 seconds; Payment: 3,600 seconds; Report: 3,600 seconds
- Personal reads
- 95% of reads are private; a shared public copy cannot answer them.
- Required proofs
- Managed encryption keys
≈ marks an approximation, just as in the game. Waiting deadlines measure the age of queued work and are separate from interactive response time.
- Payments go to a payment rail provider with its own limit.
- Payslips are stored as files.
Questions that shape the design
How fast does the payment rail accept transfers?
About 1,000 per second on your account, and sometimes it asks you to slow down to half for a while.
Does a salary have to arrive the second we send it?
No. Money may settle later that day. What matters is that each salary is paid exactly once.
Is payroll data sensitive?
Very. Salaries and bank details must be encrypted with keys held in a key service.
The traffic day
Traffic concentrated in working hours. The clock covers 00:00 to 24:00 in the home region. Events add pressure on top of that normal traffic.
02:00 · Deadline surge
Tax day, enrollment close or a sign-up deadline. Traffic climbs for hours until the cut-off.
Traffic for Read, Sign-in reaches 4 times the scheduled load; rises over 21,600 seconds, holds for 3,600 seconds and eases over 600 seconds. Other request types keep their scheduled load.
10:00 · Provider rate limit
An outside service lowers its configured request quota and answers 'too many requests' above the reduced limit.
Provider capacity falls to 50% of its configured quota. Duration: ≈ 3,600 seconds.
11:30 · Duplicate delivery
The queue redelivers messages. Without duplicate protection, side effects may be applied twice.
Affected share: 1%. Duration: ≈ 3,600 seconds.
What a strong design has to handle
- The rail can take 1,000 payments a second, and sometimes only 500. What can hold the extra payments safely until the rail is ready?
- The queue turns a payment into "accepted, will be paid today". Employees see the status change later, which is fine for salaries and wrong for a card checkout.
- Workers are sized for the average payment rate, not the burst. A bigger pool would only make the rail refuse more calls.
Make a prediction, run a full traffic day, then compare the result. The in-game editorial opens after a finished day and includes measured reference designs.
Concepts to practise
Related problems
- Easy
Invoicing and billing
Small businesses create invoices, email them to customers and take card payments online. On the last working day of the month everyone sends and chases invoices at once. Nobody may get the same invoice email twice, and no invoice may vanish.
- Medium
Buy now, pay later
At checkout a shopper picks "pay in four". You score the shopper in a few seconds, then charge the first quarter on their card. The card network sometimes asks you to slow down, and when apps retry, the retries make it worse.
- Medium
Notification service
Every other team asks you to send messages: order shipped, new login, a sale starting. You send them by push, email and text through outside providers who have their own limits. Send each message once, and never block the team that asked.
- Hard
Scheduled job service
Customers register jobs: "call my web address every night at 02:00" or "every hour on the hour". Aim to fire within 5 seconds; work still waiting after 60 seconds counts as failed. Retry safely after a crash. One external effect requires the customer endpoint to honor a stable job occurrence key. Half the world picks the top of the hour, so a huge crowd of jobs comes due in the same second.