system design · system-design

Design Netflix Open Connect (Streaming CDN)

Netflix's purpose-built CDN serving 200M+ users. Tests bitrate adaptation, edge placement, manifest design, and capacity planning. Open-ended senior-loop favorite.

expert5hawsgeneralsystem-design
Ask GPTConfidence

Theory

Explanation

Intuition first, formal definition second. Skim the bullets if you already know this; read the prose if you don't.

Streaming video is bandwidth × storage × distance. Open Connect minimizes distance: place a fleet of edge appliances (OCAs) inside ISPs so the heavy bytes flow over the last-mile, not over the public internet. Origin only ships once to each OCA; users stream from the OCA. Everything else, encoding ladder, manifest, ABR, exists to make that edge-served content adapt to each viewer's network.

Three planes: (1) Content pipeline, ingest, transcode into a per-title bitrate ladder, package as HLS/DASH segments + manifest. (2) Distribution plane, proactively fill OCAs during off-peak hours using popularity prediction; place OCAs at ISP peering points or directly inside ISP networks. (3) Client plane, player fetches manifest, runs ABR (Adaptive Bitrate) algorithm, requests segments from nearest healthy OCA. Origin services only manifest auth, license keys, recommendations, and OCA-fill traffic, not user playback.

When to use

Anytime a single piece of content is consumed by many users in many geographies (video, large game patches, OS updates, podcasts at scale). The pattern repeats: encode once → push to edge → stream adaptively.

When not to

Long-tail, low-watch content (cache miss rate too high, pay origin egress every time). Live ultra-low-latency (sub-second), HLS/DASH chunked is 2–6 s latency; use LL-HLS or WebRTC.

Time: Stream startup O(1) round-trips with prefetched manifest · Space: O(titles × ladder_rungs × geo_copies)

flowchart LR
  subgraph Content Pipeline
    Ingest[Raw Master] --> Encode{{Encoder Farm}}
    Encode --> Ladder[(Bitrate Ladder · 240p..4K HDR)]
    Ladder --> Pack[Packager · HLS/DASH]
    Pack --> Origin[(Origin Storage · S3)]
  end
  subgraph Distribution
    Predict{{Popularity Predictor}} --> Fill[Fill Scheduler]
    Origin -.off-peak fill.-> Fill
    Fill --> OCA1[(OCA · ISP-1)]
    Fill --> OCA2[(OCA · ISP-2)]
    Fill --> OCAN[(OCA · ISP-N)]
  end
  subgraph Client
    Player([Player]) --> Control[Control Plane · auth + manifest]
    Control --> Origin
    Player <-->|segments + ABR| OCA1
  end

Key insights

  • Edge placement beats edge capacity. One OCA inside an ISP serves more users at lower latency than ten OCAs in a remote datacenter.
  • Popularity prediction enables off-peak fill. New titles + trailers are pushed at 3 AM local time so daytime peak is pure-read. Misprediction cost = origin egress; over-prediction cost = wasted disk. The trade-off is tunable per market.
  • ABR ladder design is per-title, not per-system. Action films get more high-bitrate rungs; talking-head shows get lower-bitrate rungs. Per-title encoding (Netflix's 2015 innovation) cut bandwidth ~20% with no quality drop.
  • Manifest is the entire contract between origin and player. Once issued, the player can switch rungs, fail over to a backup OCA, or pause and resume, without re-hitting origin until the manifest expires.
  • Segment size is the latency knob. 2 s segments = lower latency but more requests/sec. 6 s segments = fewer requests but slower ABR reaction. Netflix uses ~4 s.
  • OCAs are simple boxes running an Nginx-derived server. Complexity lives in the fill scheduler, not the edge. That keeps fleet ops manageable.
  • Chaos engineering (Chaos Monkey) is the validation, kill an OCA, ensure clients fail over to next-nearest within the manifest's declared backup URLs.