Design group video calls

I would separate control from media. Control is a WebSocket signaling service. It handles join, the session offer and answer, and ICE candidates, and a room store maps each room to one media server using a conditional write, so everyone in a room lands together. Media goes to a selective forwarding unit.

For a 50-person call, a mesh would need 49 Mbps of upload per person, which is impossible, while an SFU keeps upload at 1.55 Mbps because each sender uploads once and the server copies packets without decoding them. An MCU would shrink client work further but costs far more computing and adds delay, so I would use it only for phone dial-in and composed recordings. Senders use simulcast: three qualities at 1, 0.4 and 0.15 Mbps. The SFU gives each receiver the speaker in the best quality that fits and the others as thumbnails, so a 9-tile view needs 2.2 Mbps.

Callers find their public address with STUN, and the 15% who cannot connect directly use TURN relays, about 37 Gbps of traffic. Lost audio is rebuilt from redundancy, and lost video is re-requested or replaced by a key frame. The main cost is media bandwidth.

These numbers are assumptions for the exercise, not facts about a real system:

• One video and audio stream at full quality is 1 Mbps. A simulcast sender also sends a 0.4 Mbps and a 0.15 Mbps copy, so 1.55 Mbps in all.
• A big call has 50 people and shows 9 tiles: one speaker at 1 Mbps and 8 thumbnails at 0.15 Mbps.
• At peak there are 20,000 calls with 4 people on average. Each person receives about 1.5 Mbps. A call lasts 30 minutes.
• A media server forwards about 600 Mbps out. 15% of callers cannot connect directly and need a relay.

The arithmetic:

Upload per person in a mesh

(N − 1) × 1 Mbps for 2, 8 and 50 people

1, 7 and 49 Mbps. At 8 it is hard, and at 50 it is impossible

Upload per person with a forwarding server

1 + 0.4 + 0.15 Mbps

1.55 Mbps at any call size

Download in a 50-person call with 9 tiles

1 + 8 × 0.15 Mbps

2.2 Mbps, instead of 49 Mbps for every stream at full quality

Media server fleet

20,000 × 4 = 80,000 people × 1.5 Mbps = 120 Gbps out; ÷ 0.6 Gbps

≈ 200 machines

Relay traffic

15% × 80,000 = 12,000 callers × (1.55 + 1.5) Mbps

≈ 37 Gbps through relay servers

Join rate

80,000 people ÷ 1,800 s per call

≈ 44 joins/s, tiny next to the media load

Signaling is the small control channel. The media protocol does not define it, so we choose a WebSocket. It carries the join request, the session offer and answer that list codecs and encryption fingerprints, and ICE candidates, which are possible addresses to try. Keep signaling servers light. Room membership lives in the room store, and servers pass messages for the same room through a publish and subscribe channel.

The NAT problem. A phone at home has a private address. Its router rewrites the address on outgoing packets and drops incoming packets from sources it has not heard from. So a caller first needs to know its public address. It asks a STUN server, a tiny service that replies with the address and port it saw. With an SFU, the media server has a public address, so the caller just sends to it, the router opens a path, and replies flow back. That is why most callers connect directly.

When that fails. Some networks block UDP outright. Others use a symmetric NAT, which picks a new public port for each destination, so the address STUN reported is useless to anyone else. Between two peers, a symmetric NAT on both sides, or a symmetric NAT facing a port-restricted one, cannot connect with STUN addresses alone. A symmetric NAT facing a full-cone or open peer often can. For the blocked cases, a TURN relay has a public address and passes media in both directions. Our estimate says 15% of callers use it, so the relay fleet carries about 37 Gbps and is costly. The caller therefore tries the candidates in order of price and keeps the first that works: a direct path, then a path using the address learned from STUN, then a relay over UDP, and finally a relay over TCP on port 443, which looks like ordinary web traffic. This ordered testing is called ICE.

Sign relay credentials with a short expiry and hand them out through signaling, so strangers cannot use the relays for free.

Room placement. The first joiner of a room makes a conditional write that maps the room to the least busy media server in the nearest region. If two people join at the same instant, only one write wins, and the other reads the winner. A mapping has a short time to live that is refreshed while the room is active. If a media server dies, clients notice that no packets arrive for about 3 seconds, rejoin through signaling, and are placed on a healthy server.

There are three topologies.

• Mesh. Every pair of callers is connected directly. Each person encodes and uploads N − 1 copies of their video. For 2 people that is 1 Mbps, for 8 it is 7 Mbps, and for 50 it is 49 Mbps, plus 49 separate encodes on one phone. There are N(N − 1) ÷ 2 links, 1,225 for 50 people. It needs no media server and has the lowest delay, so it suits calls of 2.
• SFU, a selective forwarding unit. Each person uploads one set of streams to a server, which copies packets to the others without decoding them. Upload stays at 1.55 Mbps at any size. The server's work is cheap, because it copies and rewrites packet headers, and the extra delay is one hop. This is the usual choice for groups.
• MCU, a multipoint control unit. The server decodes every stream, mixes them into one picture and one audio track, and encodes the result for each viewer. Every person uploads 1 Mbps and downloads 1 Mbps, which is easy for weak devices and phone dial-in. But the server decodes 50 streams and encodes up to 50 results, which costs far more computing than copying packets, and it adds the delay of decoding and encoding.

The arithmetic for one 50-person call on an SFU: the server receives 50 × 1.55 = 77.5 Mbps and sends 50 × 2.2 = 110 Mbps, because each viewer receives only 9 tiles. A 600 Mbps machine hosts about 5 such calls.

My choice is the SFU for groups. Use a direct mesh for two-person calls if we want lower cost, and use an MCU path only where it is needed, such as phone dial-in or a composed recording. Treat the recorder as a hidden participant that subscribes to the streams.

When a call spans regions, each caller connects to the nearest SFU, and the SFUs relay streams to each other. A stream crosses the long link once, however many people watch it on the far side.

Simulcast. The sender encodes its camera three times at once: about 720p at 1 Mbps, 360p at 0.4 Mbps and 180p at 0.15 Mbps. It uploads all three, 1.55 Mbps in total, which is 55% more than one stream. In return, the SFU can give each receiver the copy that suits it with no encoding of its own. A viewer on a weak link gets the small copy of everyone, and a viewer on fiber gets the large copy of the speaker. Layered coding in one stream does the same job with finer steps, if the codec supports it.

Choosing per receiver. The SFU estimates how much each receiver's link can carry, using feedback about packet timing, and spends that budget in order. First the audio, which is only a few tens of kilobits. Then the speaker's tile at the best layer that fits. Then the thumbnails at the low layer. As bandwidth falls, drop the speaker to the middle layer, then pause off-screen tiles, and keep the audio until last. Switch layers at a key frame, which the SFU asks the sender to produce.

The SFU also tells a sender to stop a layer that nobody is watching, which protects the sender's upload.

Loss. Packets are lost on the way. Each leg of the SFU path recovers on its own, and the round trip to the server is short, so recovery is quick.

• Audio. Use a codec with built-in redundancy that rebuilds a lost packet from the next one, and conceal what cannot be rebuilt. A short gap is better than silence.
• Video. The receiver asks for a missing packet again if the server round trip is under a few tens of milliseconds. If a frame cannot be rebuilt, it asks for a fresh key frame. Extra repair data costs 10 to 30% more bandwidth all the time, so use it only when round trips are long.
• Jitter buffer. The receiver waits 40 to 100 ms to reorder packets. A bigger buffer is smoother, and a smaller one keeps delay low. This is where the 300 ms budget is spent.

Under heavy loss, reduce the frame rate and size first, and pause video before touching the audio.