我们如何将Pinterest的广告语料库的规模扩大了60倍
Nishant Roy | Tech Lead, Ads Serving Platform
Nishant Roy | 技术负责人,广告服务平台
In May 2020, Pinterest launched a partnership with Shopify that allowed merchants to easily upload their catalogs to the Pinterest platform and create Product Pins and shopping ads. This vastly increased the number of shopping ads in our corpus available for our recommendation engine to choose from, when serving an ad on Pinterest. In order to continue to support this rapid growth, we leveraged a key-value (KV) store and some memory optimizations in Go to scale the size of our ad corpus by 60x. We had three main goals:
2020年5月,Pinterest与Shopify建立了合作关系,允许商家轻松地将他们的目录上传到Pinterest平台,并创建产品图钉和购物广告。这大大增加了我们的语料库中可供推荐引擎选择的购物广告的数量,当在Pinterest上提供广告时。为了继续支持这一快速增长,我们在Go中利用键值(KV)存储和一些内存优化,将我们的广告语料库的规模扩大了60倍。我们有三个主要目标。
- Simplify scaling our ads business without a linear increase in infrastructure costs
- 简化我们的广告业务的扩展,而无需线性增加基础设施成本
- Improve system performance
- 提高系统性能
- Minimize maintenance costs to boost developer productivity
- 最大限度地减少维护成本,提高开发人员的生产力
This blog explains how we scaled a business-critical, high traffic recommendation engine and the benefits we saw from it.
这篇博客解释了我们如何扩展一个关键业务、高流量的推荐引擎以及我们从中看到的好处。
Background
背景介绍
In 2018, here’s what the ads-serving architecture looked like:
在2018年,广告服务架构是这样的。
Fig 1: Ads-Serving Architecture in 2018
图1:2018年的广告服务架构
- An offline workflow would run every few hours to publish the index of active ads to the system. The candidate funnel service would load this index in memory. Due to memory constraints, the funnel service had nine shards.
- 一个离线工作流程将每隔几小时运行一次,向系统发布活跃广告的索引。候选漏斗服务将在内存中加载这个索引。由于内存限制,漏斗服务有九个分片。
- Downstream clients would then call the ads mixer service, which would perform feature expansion
- 然后,下游客户将调用广告混合器服务,它将进行功能扩展
- Next, the candidate retrieval service would return a list of the best candidates for the given user and context features
- 接下来,候选人检索服务将返回一个针对给定用户和背景特征的最佳候选人列表
- Then, the candidate funnel would enrich the candidates with index data, call the ranking service for scoring, and trim the candidates based on various ru...