PAUL CHONGSenior Software Engineer

WebRTC & Peer Connections

2026-08-11 · 13 min read

Most communication on the web goes through a server: your browser sends a request, the server responds. WebRTC breaks that pattern. Once two browsers have negotiated a connection, audio and video flow directly between them — no server in the media path, no round-trip delay, no server bandwidth cost for every packet.

Getting to that direct connection is the hard part. The two browsers can't just dial each other — they don't know each other's addresses, they may be behind firewalls, and they need to agree upfront on what codecs and formats they'll speak. WebRTC provides the APIs to do this. The negotiation process is verbose, but each step has a clear purpose.

The connection setup

Setting up a peer connection between Alice and Bob takes nine steps. It looks intimidating at first, but the underlying idea is simple: before the call starts, both sides agree on what they'll send and how to reach each other.

1. Alice calls getUserMedia() → gets local MediaStream (camera + microphone)
2. Alice creates RTCPeerConnection
3. Alice calls createOffer() → generates an SDP offer
4. Alice sends the SDP offer to Bob via a signaling server
5. Bob calls setRemoteDescription(offer), createAnswer(), setLocalDescription(answer)
6. Bob sends his SDP answer back to Alice via signaling
7. Both sides gather ICE candidates and send them to each other via signaling
8. ICE negotiation finds the best network path
9. Connection established — media flows peer-to-peer

Getting local media

const stream = await navigator.mediaDevices.getUserMedia({
  video: true,
  audio: true,
});

// stream contains a MediaStream with video and audio tracks
const videoEl = document.querySelector('video');
videoEl.srcObject = stream;

getUserMedia prompts the user for camera and microphone permission and returns a MediaStream. The stream is a live object — it starts capturing immediately. Tracks within it can be muted, replaced, or removed without stopping the stream.

SDP: what do we say?

SDP stands for Session Description Protocol. It's a text document — generated automatically by the browser — that describes everything about how a peer wants to communicate: which video and audio formats (codecs) it supports, what resolution and bitrate it wants, and the encryption keys for the session.

Before any audio or video can flow, both browsers need to agree on these details. They do this with an offer/answer exchange:

  • Alice's browser generates an offer — a block of text saying "here's every format I support and here's my encryption key."
  • Bob's browser reads Alice's offer, picks the formats they have in common, and generates an answer — "okay, let's use H.264 video and Opus audio, and here's my encryption key."

The end users (Alice and Bob) never see this text. App authors do handle the SDP object in code — createOffer() returns it, and you pass it to setLocalDescription() — but you treat it as an opaque blob. Your code moves it from one browser to the other; you don't read or write the contents. The browser generates it on one side and understands it on the other. But it has to travel from Alice to Bob somehow, which is what the signaling server is for.

Think of it like two people calling each other from different countries. Before the conversation starts, they quickly agree: "Can you speak English?" — "Yes, English works for me." SDP is that negotiation, but for video codecs and encryption.

// Alice's side
const pc = new RTCPeerConnection(config);
stream.getTracks().forEach(track => pc.addTrack(track, stream));

const offer = await pc.createOffer();
await pc.setLocalDescription(offer);

// Send offer.sdp to Bob via your signaling server
signalingServer.send({ type: 'offer', sdp: offer.sdp });
// Bob's side
const pc = new RTCPeerConnection(config);

// Receive offer from Alice
signalingServer.on('offer', async ({ sdp }) => {
  await pc.setRemoteDescription({ type: 'offer', sdp });

  stream.getTracks().forEach(track => pc.addTrack(track, stream));

  const answer = await pc.createAnswer();
  await pc.setLocalDescription(answer);

  signalingServer.send({ type: 'answer', sdp: answer.sdp });
});

// Alice receives the answer
signalingServer.on('answer', async ({ sdp }) => {
  await pc.setRemoteDescription({ type: 'answer', sdp });
});

ICE candidates: how do we reach each other?

SDP handles what to send. ICE (Interactive Connectivity Establishment) handles where to send it.

Both browsers are likely behind NATs or firewalls. They don't know their own public IP addresses — they know only the internal addresses their OS reports. ICE gathers a list of candidate addresses for reaching a peer and tries each one until it finds a path that works.

There are three kinds of candidates:

  • Host candidates — the machine's local network IP (works only on the same LAN)
  • Server reflexive candidates — the public IP seen by a STUN server (works across the internet when NATs allow it)
  • Relayed candidates — an address on a TURN server that relays media (works always, including corporate firewalls, but adds latency and server cost)

STUN is a simple server that tells you "here's what your public IP looks like from the outside." It's cheap and stateless. TURN is a relay — if direct connection fails, both peers send media to the TURN server and it forwards it. TURN is a fallback; you pay for its bandwidth.

const pc = new RTCPeerConnection({
  iceServers: [
    { urls: 'stun:stun.example.com:3478' },
    {
      urls: 'turn:turn.example.com:3478',
      username: 'alice',
      credential: 'secret',
    },
  ],
});

// As ICE gathers candidates, send them to the remote peer
pc.onicecandidate = ({ candidate }) => {
  if (candidate) {
    signalingServer.send({ type: 'ice-candidate', candidate });
  }
};

// Receive candidates from the remote peer
signalingServer.on('ice-candidate', async ({ candidate }) => {
  await pc.addIceCandidate(candidate);
});

ICE runs in parallel with SDP exchange. Candidates trickle in as they're discovered — you send each one to the other peer as soon as you have it rather than waiting for all candidates to be gathered.

The signaling server

The signaling server is just a message relay. It passes SDP offers, answers, and ICE candidates between peers during the handshake. Once the connection is established, the signaling server is out of the media path entirely — audio and video flow directly.

Any real-time channel works: WebSocket is the standard choice. The signaling server doesn't need to understand the messages it relays — it just needs to route them to the right room and participant.

// Minimal signaling server (Node.js + ws)
const rooms = {};

wss.on('connection', ws => {
  ws.on('message', raw => {
    const msg = JSON.parse(raw);
    const room = rooms[msg.roomId] ?? [];
    room.filter(peer => peer !== ws).forEach(peer => peer.send(raw));
    rooms[msg.roomId] = room;
  });
});

Architecture: mesh, SFU, MCU

A two-person call is straightforward — one peer connection, two participants. Group calls require choosing how participants connect to each other.

Mesh

Every participant connects directly to every other participant. A 4-person call means 6 peer connections (one for each pair). Each participant uploads their stream to every other participant.

Alice ─────────── Bob
  │ ╲           ╱ │
  │   ╲       ╱   │
  │     ╲   ╱     │
  │       ╳       │
  │     ╱   ╲     │
  │   ╱       ╲   │
  │ ╱           ╲ │
Carol ─────────── Dave

Upload bandwidth is the bottleneck. A participant sending 1 Mbps video to 4 others needs 4 Mbps of upload. With 10 participants, that's 9 Mbps — well beyond most home connections. Mesh works for 2–4 participants; it collapses past that.

SFU (Selective Forwarding Unit)

All participants send their streams to a media server. The server forwards each participant's stream to everyone else — it does not decode or mix.

UPLOAD                      DOWNLOAD

Alice ──→ SFU               SFU ──→ Bob   (Alice's stream)
                            SFU ──→ Carol (Alice's stream)
                            SFU ──→ Dave  (Alice's stream)

Bob   ──→ SFU               SFU ──→ Alice (Bob's stream)
                            SFU ──→ Carol (Bob's stream)
                            SFU ──→ Dave  (Bob's stream)

Carol ──→ SFU               SFU ──→ Alice (Carol's stream)
                            SFU ──→ Bob   (Carol's stream)
                            SFU ──→ Dave  (Carol's stream)

Dave  ──→ SFU               SFU ──→ Alice (Dave's stream)
                            SFU ──→ Bob   (Dave's stream)
                            SFU ──→ Carol (Dave's stream)

Each participant uploads one stream (to the SFU) and downloads N-1 streams. Upload cost is bounded. The SFU can also selectively forward — if Bob's window is small, the SFU sends him a lower-resolution stream without Alice needing to encode multiple qualities.

This is the architecture behind Zoom, Google Meet, and Discord video. It scales to hundreds of participants. The tradeoff is server cost — you're paying for bandwidth at the SFU.

MCU (Multipoint Control Unit)

The server decodes every participant's stream, mixes them into a single composite video (a grid), and re-encodes it. Each participant receives one stream regardless of how many people are in the call.

All participants → MCU → mixed video → each participant

Download cost is minimal (one stream per participant). Server compute cost is very high — decoding and re-encoding video is expensive. MCUs make sense for specific cases like conference room hardware or broadcast scenarios, but SFU is the more common choice for software.

Controlling media

Once the connection is established, three operations come up in every video conferencing product.

Muting

Set track.enabled = false. The track remains in the stream — no renegotiation needed — but the data it sends becomes silence (audio) or black frames (video).

function toggleMute(stream) {
  stream.getAudioTracks().forEach(track => {
    track.enabled = !track.enabled;
  });
}

This is local-only — the track is still being transmitted, just silenced. The other participant's browser receives black frames or silence rather than nothing at all. The peer connection state doesn't change.

Switching the camera

sender.replaceTrack() swaps the track on an existing peer connection without renegotiation. The remote participant seamlessly sees the new source.

// Get the current video sender
const sender = pc.getSenders().find(s => s.track?.kind === 'video');

// Get the new camera stream
const newStream = await navigator.mediaDevices.getUserMedia({
  video: { facingMode: 'environment' }, // switch to rear camera
});

const newTrack = newStream.getVideoTracks()[0];
await sender.replaceTrack(newTrack);

Screen sharing

getDisplayMedia() asks the user to pick a window or screen and returns a stream with a screen-capture track.

async function startScreenShare(pc) {
  const screenStream = await navigator.mediaDevices.getDisplayMedia({
    video: true,
    audio: true, // system audio, if the browser supports it
  });

  const screenTrack = screenStream.getVideoTracks()[0];
  const sender = pc.getSenders().find(s => s.track?.kind === 'video');
  await sender.replaceTrack(screenTrack);

  // When the user clicks "Stop sharing" in the browser UI
  screenTrack.onended = () => {
    stopScreenShare(pc, cameraStream);
  };
}

MediaStreamTrack.onended fires when the user clicks the browser's native "Stop sharing" button. Always listen for this — it's the only way to detect when the user stops sharing via the browser UI rather than your own UI.

Reconnection

Two different things can go wrong: the signaling connection or the ICE connection.

Signaling failure

The WebSocket to your signaling server drops. The peer connection itself may still be alive. Reconnect to the signaling WebSocket, fetch a room snapshot (who's currently in the room), and re-subscribe to events.

signalingSocket.onclose = async () => {
  await reconnectSignaling();
  const snapshot = await fetchRoomSnapshot(roomId);
  reconcileParticipants(snapshot);
};

ICE failure

The network path between peers breaks — the user switches from Wi-Fi to cellular, or the TURN relay goes down. pc.iceConnectionState changes to 'failed'. The cheapest fix is to try ICE restart before rebuilding the full peer connection.

pc.oniceconnectionstatechange = async () => {
  if (pc.iceConnectionState === 'failed') {
    // Try ICE restart first — reuses the existing peer connection
    await pc.restartIce();

    // If that doesn't help after a timeout, rebuild fully
    setTimeout(() => {
      if (pc.iceConnectionState !== 'connected') {
        rebuildPeerConnection();
      }
    }, 5000);
  }
};

restartIce() triggers a new round of ICE candidate gathering on the existing peer connection, which is much cheaper than tearing down and rebuilding. Rebuild only if the restart doesn't recover.

Keep browser objects out of your store

MediaStream, RTCPeerConnection, and MediaStreamTrack are browser-managed objects. They have internal state, event emitters, and lifecycle hooks. They can't be serialized — trying to put them in Redux, Zustand, or any reactive store will either throw errors or give you a stale reference.

// Don't do this
store.dispatch({
  type: 'SET_PEER_CONNECTION',
  payload: pc,  // RTCPeerConnection is not serializable
});

The right pattern is a media manager class that lives outside the store. The store holds only IDs and status flags.

class MediaManager {
  #connections = new Map(); // peerId → RTCPeerConnection

  async createConnection(peerId, config) {
    const pc = new RTCPeerConnection(config);
    this.#connections.set(peerId, pc);

    pc.oniceconnectionstatechange = () => {
      // Update the store with status only — not the object itself
      store.dispatch(updateConnectionStatus({
        peerId,
        status: pc.iceConnectionState,
      }));
    };

    return peerId; // return the ID, not the object
  }

  getConnection(peerId) {
    return this.#connections.get(peerId);
  }

  closeConnection(peerId) {
    this.#connections.get(peerId)?.close();
    this.#connections.delete(peerId);
  }
}

export const mediaManager = new MediaManager();

Components and reducers work with the peerId. When they need to act on the connection (mute, replace track), they call mediaManager.getConnection(peerId) to get the live object. The store stays serializable; the media manager handles the lifecycle.

Live broadcasting

Live streaming has two separate legs: getting the video from the streamer to a server (ingest), and getting it from that server to viewers (delivery). WebRTC only appears on the ingest side.

Ingest is one connection from one source, so the protocol just needs to be low-latency and reliable. Two options are common:

  • RTMP — the traditional choice. OBS and most streaming software use it by default.
  • WHIP (WebRTC HTTP Ingest Protocol) — a newer standard that uses a WebRTC peer connection for ingest. Sub-second latency from the streamer to the ingest server. Cloudflare Stream and Mux support it.

Delivery to viewers uses HLS or DASH — not WebRTC. The ingest server hands the stream to a transcoding server, which encodes it into multiple quality levels and chops it into short segments (2–10 seconds each). Those segments are pushed to a CDN and served to viewers via regular HTTP requests. WebRTC can't scale to 200,000 concurrent viewers; HLS/DASH can, because the CDN serves segments like static files.

The tradeoff is latency: HLS/DASH delivery typically adds 15–30 seconds of delay. Twitch's Low Latency Mode shrinks that to ~2–3 seconds using shorter segments. For sub-second interactive delivery, some platforms serve WebRTC all the way to the viewer — but at much higher server cost.

For a full breakdown of how HLS, DASH, and adaptive bitrate switching work, see Adaptive Bitrate Streaming & MSE.

Where this appears

Video conferencing — the primary use case. Zoom, Google Meet, and Discord video all use WebRTC at the peer connection layer, with SFUs for group calls. The signaling stack (room management, participant state, reconnection) is custom-built at each company, but the media layer is WebRTC.

Live audio — Discord's voice channels, Clubhouse, Twitter Spaces. Audio-only connections are the same API with video tracks omitted. Lower bandwidth requirements make mesh viable for small groups.

Screen sharing and remote desktop — getDisplayMedia() is used in collaborative tools (Figma's multiplayer observe mode, Loom-style screen recording), and remote desktop tools (thinclients that stream a desktop session over WebRTC data channels).

Data channels — RTCDataChannel is a peer-to-peer data pipe with no audio or video involved. Used for multiplayer game state (low-latency position updates), file transfer between browsers, and collaborative whiteboard state that needs sub-100ms delivery.

Live streaming — WebRTC appears on the ingest side (streamer → server) via WHIP. Delivery to viewers uses HLS/DASH, not WebRTC — the viewer count is too large for individual peer connections. See Adaptive Bitrate Streaming & MSE for how the delivery side works.