Design a video streaming service

In 2 minutes

I would build two paths that meet only at storage. For uploads, the client sends resumable pieces to an upload service, which stores the original and queues a job. Workers split the video into 4-second segments at keyframes and encode every segment in parallel across a bitrate ladder, then write a manifest. In our estimate that is about 8,300 cores continuously, and a one-hour video is ready in roughly a minute. For playback, I would use HLS or DASH over plain HTTP. The player gets a manifest, then fetches one segment at a time, and picks the rendition for the next segment from its measured speed and buffer, switching at segment edges. Because segments are immutable files, a CDN can cache them near viewers for a long time. At 5 million streams and 3 Mbit/s, we need 15 Tbit/s, and with a 98 percent hit rate the origin only has to supply about 300 Gbit/s. A regional cache tier collapses misses. Popular and new titles are pre-loaded, and the long tail is pulled through on demand. The cost is a slower first play of a rare video.

The question

Clarifying questions

• Is this on-demand video only, or live streaming too? Live needs a continuous ingest and a manifest that keeps growing. Starting with on-demand keeps the first design clear.
• How much video is uploaded each day, and how long are the videos? Hours uploaded decide how much encoding capacity you need, and how soon a video can be watched after upload.
• How many people watch at once at the peak, and where are they? Peak viewers multiplied by bitrate is the bandwidth bill, and their location decides where the caches must be.
• Which devices must we support? The oldest device sets the codec and format that every video must also be available in.
• Do we need protection against copying, such as encrypted streams? Encryption and licence servers add steps to both the encoding pipeline and the player.

Requirements

Functional

• Upload a video, including large files over unreliable connections.
• Process each upload into formats and sizes that play on every supported device.
• Stream a video with a quality that adapts to the viewer's connection.
• Let people seek to any point, and resume where they stopped.

Non-functional

• Playback starts in under 2 seconds at p95.
• Rebuffering stays rare: fewer than 1 in 100 viewing minutes interrupted.
• A new upload is watchable within minutes of finishing, not hours.
• Handles 5 million concurrent streams at the peak.

Back-of-the-envelope estimates

Assumptions for this exercise:

• These numbers are assumptions for the exercise, not facts about any real service.
• 50,000 hours of video are uploaded a day at an average of 8 Mbit/s (1 MB/s). All the encoded sizes together take about as much space as the original.
• Encoding the full ladder of sizes takes 4 core-hours per hour of video. Segments are 4 seconds long.
• 5 million streams at the peak, averaging 3 Mbit/s. A CDN serves 98% of the bytes from its caches.

The arithmetic:

• New uploads per day: 50,000 hours × 3,600 s × 1 MB/s = 180,000 GB → 180 TB a day of originals
• New encoded storage per year: 180 TB a day × 365 → ≈ 65.7 PB
• Encoding capacity: 50,000 hours × 4 core-hours = 200,000 core-hours a day ÷ 24 → ≈ 8,300 cores running all day
• Speed for a 1-hour video: 3,600 s ÷ 4 s = 900 segments. 4 core-hours = 14,400 core-seconds ÷ 900 → 16 core-seconds per segment, so about a minute if the 900 segments run in parallel
• Peak playback bandwidth: 5M streams × 3 Mbit/s = 15,000,000 Mbit/s → 15 Tbit/s
• Traffic reaching the origin: 15 Tbit/s × 2% not served by CDN caches → 300 Gbit/s

High-level design

There are two separate paths. The upload path starts with an Uploader who sends the file through the API gateway to the Upload service. The service writes the original to Video storage and puts a job on the Transcode queue. Transcode workers take the job, cut the video into short segments, encode each segment in several sizes in parallel, and write the results back to storage with a manifest that lists them.

The playback path starts with a Viewer. The player asks the CDN for a manifest, then for one segment at a time. Popular segments are already in the CDN's caches near the viewer. A miss goes to the Origin, which reads Video storage.

The two paths meet only at storage, so a surge of viewers never slows encoding, and a backlog of uploads never slows playback.

How a request flows

Upload and transcode

1. The creator uploads the video in small pieces, so an interrupted upload can resume from the last piece.
2. The gateway checks the creator's identity and routes the pieces to the upload service.
3. The service assembles the pieces and writes the original file to video storage.
4. It then queues a transcode job holding only the video ID.
5. A coordinator worker takes the job and splits the video into 4-second segments. Many workers then each encode some segments.
6. Each segment is encoded in several sizes and written to storage. Last, the manifest listing them is written.

Watch a popular video

1. The player asks for the manifest, a small text file that lists the sizes and the segment addresses.
2. The nearest CDN cache already holds it, because many people watch this title, and returns it at once.
3. The player measures its download speed and asks for the next segment at a quality it can sustain.
4. The segment comes from the cache too. The origin and storage are never touched.

Watch a rarely played video

1. A viewer opens an old video that few people watch and asks the CDN for a segment.
2. The cache has no copy, so it asks the origin.
3. The origin reads that segment from video storage.
4. Storage returns the segment bytes.
5. The origin returns the segment, and the CDN keeps a copy for the next viewer.
6. The player receives the segment. This first request is slower, but later ones are cached.

Deep dive: upload and the transcoding pipeline

Uploads can be many gigabytes, so make them resumable: the client sends fixed-size pieces, the server records which arrived, and after a failure the client continues from the first missing piece. Writing the original to storage first means the later steps can fail and retry without asking the creator to upload again.

Raw uploads are too large and too slow for most viewers, so we transcode: convert the video into several sizes and bitrates, called a bitrate ladder, for example 240p, 480p, 720p, 1080p and 4K, each at a bitrate suited to its size. We also encode audio as its own track.

Encoding one two-hour file from start to finish takes hours. The trick is to split first:

1. Split the video at keyframe boundaries into segments of a few seconds. Each segment can be decoded on its own.
2. Encode in parallel. Every segment becomes a small task per rendition. Thousands of tasks spread over a pool of workers, so a one-hour video finishes in about a minute in our estimate.
3. Assemble the manifest that lists every rendition and segment.

Make tasks idempotent and small. If a worker dies, only its segment is retried. Track job state in a database so a restart resumes where it stopped. Publish early: once the lowest renditions are ready, the video can go live while the larger ones are still being encoded.

Codec choice matters. A widely supported codec such as H.264 plays on nearly every device. Newer codecs make smaller files at the same quality, but cost far more CPU and need newer devices. A sensible plan is to encode everything in the compatible codec and re-encode only popular titles in the newer one, once the bandwidth saved justifies the extra encoding. Run content checks, such as copyright matching, as separate tasks, not inside the critical path.

Deep dive: adaptive bitrate with manifests and segments

Modern streaming uses plain HTTP. The video is split into short segments, each a small file, and a manifest describes them. The two common formats are HLS, whose manifest is a text playlist, and DASH, whose manifest is an XML file. Both work the same way in principle: the manifest lists each rendition and the addresses of its segments, in order.

The player does the thinking. It downloads a few segments, measures how fast they arrived and how full its buffer is, and then picks the rendition for the next segment. This is adaptive bitrate streaming.

• If the buffer is low or the speed drops, switch to a smaller rendition. A blurry picture is better than a stall.
• If the speed is comfortably above the next rendition's bitrate and the buffer is healthy, switch up.
• Start on a lower rendition to begin playing in under two seconds, then climb.

All renditions use the same segment boundaries, so the player can switch at any segment edge without a visible glitch. Seeking is easy: compute which segment holds the target time and fetch it.

Why plain HTTP? Segments are ordinary files, so any web cache or CDN can store them, they pass through firewalls, and there are no special streaming servers to run. The server does not keep a session per viewer, which is what lets one title serve millions.

Live streaming uses the same idea with shorter segments and a manifest that keeps growing. Latency is a few segments long, which is why live over HTTP is often 5 to 30 seconds behind.

Deep dive: CDN for popular titles, origin for the long tail

At peak, 5 million streams at 3 Mbit/s need 15 Tbit/s. No single data centre serves that, and the distance would hurt quality. A content delivery network places caches close to viewers, often inside internet providers' networks. The player's requests go to the nearest cache, and our own servers see only the misses.

Viewing is very uneven. A small share of titles gets most of the views, and a long tail is rarely played. This is why caching works so well: with a hit rate of 98% by bytes, the origin handles 300 Gbit/s, one fiftieth of the total.

Segments are immutable. A segment's address never changes its content, so caches can keep it for a long time with no invalidation. A re-encoded video gets a new address. Manifests of finished videos can be cached for long too. Only a live manifest needs a short lifetime.

Use tiers. A miss at a local cache goes first to a regional cache and only then to the origin. When a million people start the same new episode, the regional tier collapses their misses into one origin request per segment.

Handle the two ends differently.

• Popular and new titles. Push copies to the caches ahead of a big release, so the first viewers do not all miss together.
• The long tail. Do not pre-load. Let the first viewer pull the segment through, and accept a slower start. Keep only a few renditions of rarely watched videos, and move their files to cheaper storage.

The player can help too, by fetching a couple of segments ahead of the playhead, so a slow cache miss hides behind the buffer.

Trade-offs

• Transcoding: Split into segments and encode in parallel over Encode each full file on one machine. A long video is ready in minutes and a failure costs one segment. The cost is the splitting and assembling steps, and slightly less efficient compression at segment edges.
• Delivery: Adaptive bitrate over HTTP segments over A custom streaming protocol with one bitrate. It adapts to each viewer's speed and uses ordinary CDNs and caches. The cost is encoding and storing many renditions.
• Codecs: Compatible codec for all, newer codec for popular titles over Newest codec for everything. Every device can play every video, and bandwidth is saved where it counts. The cost is a second encode for popular titles.
• Cache fill: Pull-through, with pre-loading for big releases over Push every video to every cache. Caches hold what people actually watch. The cost is a slower first play of a rare video.

Common follow-up questions

How would you add live streaming?
Accept a continuous stream from the broadcaster, cut it into segments of two to six seconds as it arrives, and keep appending them to a manifest that players refetch every few seconds. Encoding must keep up in real time, so you encode per stream and not per segment. Expect several seconds of delay.

What happens when one video goes viral?
The CDN absorbs it, since every segment is a cacheable file. The tiered caches keep the number of origin requests low by merging simultaneous misses. If it was predictable, such as a premiere, pre-load the first segments into the caches.

How would you protect videos from copying?
Encrypt segments, and have the player fetch a key from a licence server after checking the viewer's rights. Add a fingerprint check at upload to catch known copyrighted material. No scheme is perfect, so combine technical limits with monitoring.

How do you keep storage costs under control?
Measure views per title. Keep the full ladder for popular videos on fast storage. For the long tail, drop rarely used renditions and move the rest to cheaper, slower storage, and re-create them on demand if viewing picks up.