使用大型语言模型提高Pinterest搜索相关性

Han Wang | Machine Learning Engineer, Relevance & Query Understanding; Mukuntha Narayanan | Machine Learning Engineer, Relevance & Query Understanding; Onur Gungor | (former) Machine Learning Engineer, Relevance & Query Understanding; Jinfeng Rao | Machine Learning Engineer, Pinner Discovery

Han Wang | 机器学习工程师，相关性与查询理解；Mukuntha Narayanan | 机器学习工程师，相关性与查询理解；Onur Gungor | （前）机器学习工程师，相关性与查询理解；Jinfeng Rao | 机器学习工程师，Pinner发现

Figure: Illustration of the search relevance system at Pinterest.

图：Pinterest上搜索相关性系统的插图。

Background

背景

Pinterest Search is one of the key surfaces on Pinterest where users can discover inspiring content that aligns with their information needs. Search relevance measures how well the search results aligned with the search query. Using a relevance objective allows the search engine to ensure that the content displayed to users is genuinely pertinent to their information needs, rather than overly relying on factors like past user engagement.

Pinterest搜索是Pinterest上的一个关键界面，用户可以在此发现与其信息需求相符的启发性内容。搜索相关性衡量搜索结果与搜索查询的对齐程度。使用相关性目标可以确保搜索引擎向用户展示的内容确实与他们的信息需求相关，而不是过度依赖过去用户参与等因素。

In this work, we focus on improving the search relevance model. To measure the relevance between queries and Pins, we use a 5-level guideline (see Table 1).

在这项工作中，我们专注于改善搜索相关性模型。为了衡量查询与 Pins 之间的相关性，我们使用 5 级指南（见表 1）。

Table 1: 5-scale Pin relevance guidelines.

表1：5级Pin相关性指南。

In this blog, we will go through the technical design and share some offline and online results for our LLM-based search relevance pipeline. More details can be found in our full paper.

在本博客中，我们将介绍技术设计，并分享一些我们基于LLM的搜索相关性管道的离线和在线结果。更多细节可以在我们的完整论文中找到。

Technical Design

技术设计

LLM as Relevance Model

LLM 作为相关性模型

We use a cross-encoder language model to predict a Pin’s relevance to a query, along with Pin text, as shown in Figure 1. The task is formulated as a multiclass classification problem. We fine-tune the models using human-annotated data, minimizing cross-entropy loss.

我们使用交叉编码器语言模型来预测一个Pin与查询的相关性，以及Pin文本，如图1所示。该任务被表述为一个多类分类问题。我们使用人工标注的数据对模型进行微调，最小化交叉熵损失。

Figure 1: The c...