使用 JDK’s Vector API 优化推荐系统

Ranker is one of the largest and most complex services at Netflix. Among many things, it powers the personalized rows you see on the Netflix homepage, and runs at an enormous scale. When we looked at CPU profiles for this service, one feature kept standing out: video serendipity scoring — the logic that answers a simple question:

Ranker 是 Netflix 最大和最复杂的服务之一。其中许多功能包括驱动你在 Netflix 主页上看到的个性化行，并且以巨大的规模运行。当我们查看该服务的 CPU 配置文件时，一个特性一直脱颖而出：video serendipity scoring — 回答一个简单问题的逻辑：

“How different is this new title from what you’ve been watching so far?”

“这个新标题与您迄今为止观看的内容有多大不同？”

This single feature was consuming about 7.5% of total CPU on each node running the service. What started as a simple idea — “just batch the video scoring feature” — turned into a deeper optimization journey. Along the way we introduced batching, re-architected memory layout and tried various libraries to handle the scoring kernels.

这个单一功能消耗了运行该服务的每个节点上约 7.5% 的总 CPU。原本一个简单的想法—“just batch the video scoring feature”——演变成了一个更深入的优化之旅。一路上，我们引入了 batching，重构了内存布局，并尝试了各种库来处理 scoring kernels。

Read on to learn how we achieved the same serendipity scores, but at a meaningfully lower CPU per request, resulting in a reduced cluster footprint.

继续阅读，了解我们如何实现相同的 serendipity scores，但每个请求的 CPU 消耗显著降低，从而减少了集群 footprint。

Problem: The Hotspot in Ranker

问题：Ranker 中的热点

At a high level, serendipity scoring works like this: A candidate title and each item in a member’s viewing history are represented as embeddings in a vector space. For each candidate, we compute its similarity against the history embeddings, find the maximum similarity, and convert that into a “novelty” score. That score becomes an input feature to the downstream recommendation logic.

总体而言，serendipity scoring 的工作方式如下：候选标题和成员观看历史中的每个项目都被表示为向量空间中的 embeddings。对于每个候选，我们计算其与 history embeddings 的 similarity，找到最大 similarity，并将其转换为“novelty” score。该 score 成为下游 recommendation logic 的输入特征。

The original implementation was straightforward but expensive. For each candidate we fetch its embeddin...