eBay速度惊人的十亿级矢量相似性引擎

Often, ecommerce marketplaces provide buyers with listings similar to those previously visited by the buyer, as well as a personalized shopping experience based on profiles, past shopping histories and behavior signals such as clicks, views and additions to cart. These are vital to the shopping experience, and so it’s equally vital that we continuously improve accuracy and robustness in scalability and performance.

通常情况下，电子商务市场为买家提供与买家之前访问过的商品类似的列表，以及基于个人资料、过去的购物历史和行为信号（如点击、查看和添加到购物车）的个性化购物体验。这些对购物体验至关重要，因此，我们不断提高可扩展性和性能的准确性和稳健性同样至关重要。

eBay has approximately 134 million users and 1.7 billion live listings at any given time on the marketplace, as of December 2022. Recently, the eBay CoreAI team launched an "Approximate Nearest Neighbor" (ANN) vector similarity engine that provides tooling to build use cases that match semantically similar items and personalize recommendations. More specifically, given an input listing, the similarity engine finds the most similar listings based on listing attributes (title, aspect, image) for item-to-item similarity or generates personalized listing recommendations based on a user’s past browsing activity for user-to-item objective.

截至2022年12月，eBay在市场上任何时候都有大约1.34亿用户和17亿条实时列表。最近，eBay CoreAI团队推出了一个 "近似近邻"（ANN）矢量相似性引擎，该引擎提供工具来构建使用案例，以匹配语义相似的项目并进行个性化推荐。更具体地说，给定一个输入列表，相似性引擎根据列表属性（标题、方面、图像）找到最相似的列表，以实现项目对项目的相似性，或者根据用户过去的浏览活动生成个性化的列表推荐，以实现用户对项目的目标。

This article takes a look at the architecture we constructed for vector-based similarity, meeting the scale using data sharding, partitioning and replication, and features including attribute-based features and a pluggable ANN backend based on index building, recall accuracy, latencies and memory footprint.

本文介绍了我们为基于矢量的相似性构建的架构，利用数据分片、分区和复制来满足规模，以及包括基于属性的特征和基于索引构建的可插拔的ANN后端、召回精度、延迟和内存占用的特征。

What is Vector Similarity?

什么是矢量相似性？

Keyword-based retrieval methods often struggle with certain challenges. Among those are words that have a dual meaning (otherwise known as homonyms); more targeted keyword phrases in which users add a great deal of deta...