Design an email delivery service

In 2 minutes

I would accept mail through a thin API and do the real work behind a queue. The API checks the key and the suppression list, stores the message, queues a small job with its ID, and returns 202 with a message ID. Jobs sit in lanes by recipient domain, so one slow or strict provider never blocks the rest, and each lane has its own rate that adapts to the replies it gets. Sender workers load the message, sign it with the customer's DKIM key and send it over SMTP. A 2xx reply is final, a 4xx is retried with growing delays for about three days, and a 5xx is a hard bounce. Bounces and complaints come back through a processor that feeds the suppression list, so we never mail those addresses again. Deliverability rests on SPF, DKIM and DMARC records for each customer, and on reputation. I would keep transactional and marketing mail on separate IP pools and warm new IPs by doubling volume daily. Clicks are tracked by signed redirect links, while opens are treated as a rough signal, since image proxies and blockers distort them. Customers get status events by webhook.

The question

Clarifying questions

• What kinds of mail do customers send: transactional, marketing, or both? Transactional mail such as a password reset must not share a sending reputation with bulk campaigns.
• How many emails a day, and how large are the biggest campaigns? A campaign to millions of addresses at one mail provider is limited by that provider's acceptance rate, which decides how long it takes.
• Do customers need to know what happened to each message? Status events, such as delivered, bounced and clicked, need a webhook feed and a place to keep the history.
• Do we send from the customer's own domain or from ours? Sending from the customer's domain needs DKIM keys per customer and DNS records they must add, and it ties reputation to that domain.
• Do we track opens and clicks? Tracking adds a public endpoint, link rewriting and privacy duties, and the numbers it gives are not fully reliable.

Requirements

Functional

• Accept a message through an API and deliver it to the recipient's mail server.
• Retry temporary failures and stop on permanent ones.
• Process bounces and spam complaints, and never mail an address that has asked us to stop.
• Report a status for each message to the customer, by API and webhook.

Non-functional

• Transactional mail leaves our servers within 10 seconds of the API call at p99.
• No accepted message is lost, even if a worker or a machine dies.
• Keeps bounce and complaint rates low enough to protect sending reputation.
• Handles 2,900 messages a second at peak, and campaigns of 5 million.

Back-of-the-envelope estimates

Assumptions for this exercise:

• These numbers are assumptions for the exercise, not facts about any real service.
• 50 million emails a day, with a peak 5 times the average. An average message is 30 KB with its HTML.
• A large receiving mail provider accepts about 50 messages a second from one sending IP address.
• A new IP address is warmed up by sending 200 messages on day one and doubling the volume each day.

The arithmetic:

• Messages per second: 50M a day ÷ 86,400 s ≈ 579 → ≈ 580/s on average, about 2,900/s at a 5x peak
• Peak outbound bandwidth: 2,900 × 30 KB = 87 MB/s × 8 bits → ≈ 700 Mbit/s
• Message storage: 50M × 30 KB = 1.5 TB a day × 30 days → ≈ 45 TB
• A 5-million campaign to one big provider: 5M ÷ 50 a second = 100,000 s ÷ 3,600; with 10 IP addresses, a tenth of that → ≈ 28 hours on one IP, ≈ 2.8 hours on ten
• Warming up a new IP: 200 × 2^n reaches 5M when 2^n ≈ 25,000, so n ≈ 15 → About 15 days of doubling

High-level design

A Customer app calls the Send API. The API checks the key, validates the message and checks the recipient against the Suppression list. It stores the message in the Message store and puts a small job on the Outbound queue. It then answers 202 Accepted with a message ID.

The queue is split into lanes by recipient domain. Sender workers take jobs that are due, load the message, sign it with DKIM and open an SMTP connection to the Receiving mail servers. Each lane has its own speed limit, so a slow or strict provider cannot hold up the others.

Results come back in two ways. The immediate SMTP reply says accepted, temporarily refused or permanently refused. Later, Bounce reports arrive from receivers, and a Bounce processor turns them into entries on the suppression list, so we never try those addresses again.

How a request flows

Send an email

1. The customer's app calls the send API with the recipient, subject, body and a unique request key.
2. The API checks that the address is not on the suppression list.
3. The address is clean, so the message can go ahead.
4. The API saves the message and sets its status to queued.
5. It puts a small job holding just the message ID on the lane for the recipient's domain.
6. The API replies 202 Accepted with the message ID. The customer does not wait for delivery.

Deliver and retry

1. A sender worker takes the next job from a lane that is allowed to send now.
2. The worker asks the message store for the full message.
3. The store returns the message and the worker signs it with the customer's DKIM key.
4. The worker opens an SMTP connection to the recipient's mail server and sends the message.
5. The server replies: 2xx accepted, 4xx try later, or 5xx never.
6. On a 4xx, the job goes back to the lane with a longer delay. On 2xx the job is acknowledged and the status updated.

Handle a bounce

1. A receiving server later reports that the address does not exist, in a bounce message to our return address.
2. The processor matches the bounce to the original message and adds the address to the suppression list.
3. From then on, the API finds the address on the list and refuses to queue mail to it.

Deep dive: the send pipeline: lanes, throttling and retries

Email moves by SMTP. A worker looks up the recipient domain's mail servers in DNS, using its MX records, opens a connection, and walks through a short dialogue: who the message is from, who it is for, then the content. Every step gets a reply code. A 2xx code means accepted. A 4xx code means a temporary refusal, such as a full mailbox, a rate limit or greylisting. A 5xx code means a permanent refusal, such as an address that does not exist.

Queue the work in lanes by recipient domain. If one provider is slow or blocking us and all mail shares a single queue, every other message waits behind it. This is head-of-line blocking. Each lane gets its own concurrency limit and send rate, and it adapts: slow down when the provider replies with rate-limit errors, and speed up gradually when it accepts everything.

Reuse connections. Sending many messages down one open connection is much cheaper than a new handshake each time, and providers prefer it.

Retries. Temporary failures retry with growing delays and random jitter, such as 1 minute, 5 minutes, 30 minutes, 2 hours, then every 6 hours for up to about three days. After that, report the message as failed. Permanent failures are never retried.

Bounces. A hard bounce is a 5xx on the address: add it to the suppression list at once. Some receivers accept a message first and reject it later by sending a bounce email. To match it with the original, give each message its own return address with an encoded ID, so a processor knows which message and customer it belongs to. Spam complaints reach us through feedback loops that the large providers offer, and those addresses are suppressed too.

Report every state change (queued, delivered, deferred, bounced) to the customer through webhooks.

Deep dive: deliverability: SPF, DKIM, DMARC and reputation

Receivers want proof that the mail is really from who it claims. Three DNS-based checks work together.

• SPF. A DNS record for the sending domain lists the servers allowed to send for it. The receiver compares the connecting IP address with that list.
• DKIM. The sender signs parts of each message with a private key. The receiver finds the matching public key in DNS and checks the signature, which shows the message was authorised by the domain and not changed on the way.
• DMARC. A DNS policy that says what to do when a message fails both checks for the visible From domain: nothing, quarantine it or reject it. It also names an address for reports.

For a service that sends on behalf of customers, the customer adds our SPF include and a DKIM key record to their DNS. We sign each message with that customer's key, so their mail passes authentication and the reputation attaches to their own domain.

Reputation decides the rest. Receivers judge each IP address and domain by bounce rates, spam complaints, spam-trap hits and whether people engage. Keep bounces and complaints very low. Never mail addresses on the suppression list, include a List-Unsubscribe header on bulk mail, and honour opt-outs fast.

IP warm-up. A new IP address has no history, and a sudden flood from it looks like spam. Start small, with your most engaged recipients, and roughly double the volume each day. From 200 messages, 15 doublings reach about 6.5 million.

Separate IP pools: one for transactional mail and another for marketing. A complaint spike from a newsletter then never delays a password reset.

Deep dive: open and click tracking, and its limits

Clicks. Rewrite each link to point to a tracking address on a domain you control, with a signed token that stands for the original link. When the recipient clicks, the tracking endpoint records the event and answers with a 302 redirect to the real destination. Use a 302 rather than a 301, so every click reaches you. Sign the token so nobody can edit it into a redirect to a site of their choice.

Opens. Add a one-pixel image whose URL carries a message token. If the recipient's mail app loads images, the request tells you the message was opened.

Treat these numbers carefully.

• Many mail apps block remote images by default, so real opens are undercounted.
• Some providers load images through their own proxy as soon as mail arrives, so opens are overcounted and the location and time are the proxy's, not the person's.
• Security scanners follow links in incoming mail, which produces clicks from machines. Filter obvious bot patterns, such as many clicks within seconds of delivery.
• A scanner could also trigger an unsubscribe link. Make the link open a confirmation page, so the opt-out needs a real human action.

So opens are a rough signal, and clicks are better but not perfect. Never base an important decision, like removing a subscriber, on opens alone.

Privacy. Tracking reveals when and roughly where a person reads mail. Let customers turn it off per message or per domain, honour consent rules where they apply, keep only the data you need, trim IP addresses, and set a retention limit. A dedicated tracking domain per customer keeps tracking links from harming the shared domain's reputation.

Trade-offs

• Queue layout: A lane per recipient domain over One shared queue. A slow or strict provider cannot block everyone else, and each lane can follow its own speed limit. The cost is many lanes to run and monitor.
• Retry window: Retry for about three days, with growing delays over Fail after a few quick attempts. Most temporary problems clear within hours, so far fewer messages are lost. The cost is that a truly undeliverable message takes days to report.
• Sending IPs: Separate pools for transactional and marketing mail over One shared pool. Marketing complaints cannot hurt password-reset delivery. The cost is more IP addresses to warm up and pay for.
• Tracking: Link tracking on by default, open tracking optional over Track everything always. Clicks are more reliable and less invasive than opens. The cost is a less complete picture for customers who want open data.

Common follow-up questions

What do you do when a big provider starts blocking you?
Read the rejection codes, slow that lane down and check whether a list or a customer is the cause. Move the traffic to a better-reputed IP pool if you have one. Then fix the source, usually by cleaning a list or suspending a sender, before raising the rate again.

How do you stop customers from sending spam through you?
Verify new accounts, start them with low limits and review content and recipient lists on the way. Watch each customer's bounce and complaint rates, and pause sending automatically above a threshold. One bad customer can harm a shared IP for everyone.

Can you guarantee a message is delivered exactly once?
No. The queue is at-least-once, and the final step to a remote server can fail after the server has accepted the message. An idempotency key on the API stops duplicate requests, and a message ID stops most duplicate sends. A rare repeat is the price of never losing a message.

How would a customer know about each status change?
Store an event for each change and push it to the customer's webhook, with retries and a signature they can check. Keep the history queryable through the API for a limited time, so customers can look up a single message.