How Netflix Decides What to Show You (And Why It's Not Just ML)

You open the app, and inside a couple hundred milliseconds you’re looking at rows that feel eerily specific — a “Because you watched” shelf with the right show in it, a thumbnail for the same movie your friend is watching that looks nothing like the one you’d get. That’s not one model. That’s a pipeline making dozens of decisions per request, at a scale where the wrong architecture doesn’t just produce bad recommendations — it produces a page that doesn’t load in time.

At Netflix’s scale — hundreds of millions of member profiles, a catalog in the low thousands of titles, and a latency budget measured in milliseconds — “recommend the right thing” is really three separate systems glued together, each with its own failure modes.

The Problem, Properly Framed

Recommendation at this scale isn’t a ranking problem, it’s a ranking problem under a latency budget, with a catalog small enough that you can’t hide behind volume the way a search engine can. When someone opens the home page, the system has roughly 100-200ms to decide, per row, per profile, what to show — and it has to make that decision from a catalog that’s genuinely finite, which means the same handful of hit titles get evaluated against every one of hundreds of millions of profiles constantly. Get the ranking wrong for a popular title and you don’t dilute the damage across a long tail, you concentrate it.

The naive version — a single model that scores every title for every user in real time — falls over on both axes: scoring the full catalog per request blows the latency budget, and a monolithic model can’t easily encode both “similar to what you’ve watched” and “trending in your region this week” and “your specific device’s decoder constraints” without turning into an unmaintainable feature-engineering swamp.

The Core Mechanics

Candidate Generation: cut the catalog before you rank it

The first stage isn’t ranking, it’s elimination. Given a profile, a set of lightweight models — collaborative filtering on watch history, content-based similarity on metadata and embeddings — narrows the full catalog down to a few hundred candidates per row, precomputed on a batch cadence rather than at request time. This is the design decision that makes the latency budget survivable: expensive computation happens offline, hours before anyone opens the app, and the online path only ever touches a small candidate set.

# simplified candidate generation, batch job
def generate_candidates(profile_embedding, catalog_embeddings, k=300):
    # cosine similarity against a precomputed catalog index
    scores = catalog_embeddings.dot(profile_embedding)
    top_k = np.argpartition(scores, -k)[-k:]
    return top_k  # written to a feature store, keyed by profile_id

The trade-off here is staleness. A batch job that runs nightly means a title you watched an hour ago doesn’t influence candidate generation until tomorrow’s run — which is why this stage alone isn’t the whole system.

Ranking: the expensive model runs on a cheap set

Once the candidate set is small, a heavier model — gradient-boosted trees or a learned ranker — scores those few hundred candidates against dozens of features: recency of similar watches, time of day, device type, completion rates for similar content, even how well a thumbnail variant has historically performed for people with a similar profile. This is where the real personalization happens, and it’s affordable specifically because the candidate stage already did the expensive elimination.

Real-Time Signals: closing the gap batch can’t

The candidate set is stale by design, so a separate real-time layer patches it: if you stopped a show 10 minutes ago, or just searched for a specific actor, that signal needs to influence the ranking on your very next page load, not tomorrow’s batch run. This layer is deliberately narrow — a small set of session features merged into the ranking stage at request time — because trying to make the whole pipeline real-time would reintroduce the latency problem the batch/online split was built to solve in the first place.

The Trade-offs / What Goes Wrong

Cold start is a real, unsolved cost, not an edge case. A new profile has no watch history, so collaborative filtering has nothing to work with — the system falls back to popularity and coarse demographic priors, which means new users see a less personalized, more generic experience for weeks, not seconds. There’s no clean fix here, only a slower ramp.

Feedback loops concentrate popularity. If the ranker is trained on what people clicked, and what people clicked was shaped by what the previous model ranked highly, you get a self-reinforcing loop where already-popular titles get more exposure, get more clicks, and get ranked even higher next cycle — at the expense of a title that might have performed just as well if it had ever been shown. Netflix mitigates this with deliberate exploration (showing a small percentage of traffic titles outside the model’s top ranking, à la a multi-armed bandit), which costs short-term engagement to protect long-term catalog utilization.

Batch/online staleness is a permanent tension, not a bug to fix once. Every hour you extend the batch cadence, you save compute cost and add staleness. Every hour you shrink it, you burn more infrastructure spend for freshness that most sessions won’t even notice. There’s no setting that’s correct for both a binge-watcher mid-session and someone who opens the app twice a month — this trade-off gets tuned, not solved.

At p99.9, the latency budget is the thing that breaks, not the model quality. During peak hours, the tail latency on the real-time ranking call is what determines whether a row renders on time or falls back to a cached, less personalized default. A model that’s 2% more accurate but adds 40ms to p99 latency is often the wrong trade — a slightly worse recommendation that loads on time beats a slightly better one that times out into a generic fallback.

Takeaways

• The candidate generation / ranking split exists to survive a latency budget, not because it’s architecturally elegant — expensive computation gets pushed offline specifically so the online path stays cheap.
• Real-time signals are narrow by design. Making the whole pipeline real-time would just reintroduce the latency problem the batch split was built to solve.
• Feedback loops are an active risk in any system trained on its own output, and deliberate exploration is the standard mitigation, at a measurable short-term cost.
• Staleness is a dial, not a defect — batch cadence gets tuned against infrastructure cost, and there’s no cadence that’s simultaneously optimal for every usage pattern.
• At real scale, tail latency competes directly with model quality for the same decision, and the right call is often the faster, slightly-worse answer.

If you found this useful or want to discuss it further, connect with me on GitHub or LinkedIn.