As more use cases are onboarded to Manas, one special scalability and efficiency challenge emerges when serving documents with a hierarchical structure. Manas, as a traditional search engine, was designed and optimized to support flattened documents. As a result, we have to flatten attributes of root documents all the way to the leaf level regardless of the hierarchical structure, leading to inefficiencies in both indexing and serving pipelines.

在上一篇文章中，我们介绍了我们的内部搜索引擎--Manas，并分享了我们如何大规模地提供基于术语的搜索服务。自推出以来，Manas已经成长为Pinterest的关键候选生成器之一，服务于许多超出其最初目的的用例。

特别是，基于嵌入的检索是Pinterest的发现和推荐引擎的一个关键组成部分。Manas传统上支持通过位置敏感哈希(LSH)在反向索引上进行近似最近邻(ANN)搜索，这是基于术语搜索引擎的自然扩展。在Hierarchical Navigable Small World graphs (HNSW)等新的先进技术发布后，我们在Manas中建立了一个灵活的基于嵌入的检索框架，使我们能够轻松地搭载新的ANN技术。我们使用新的框架向我们的批量索引集群推出了HNSW（从几分钟到几天的索引延迟），与LSH相比，我们节省了巨大的服务成本，降低了延迟。

Manas是Pinterest内部的搜索引擎，是一个通用的信息检索平台。Manas被设计为一个具有高性能、可用性和可扩展性的搜索框架。今天，Manas为大多数Pinterest产品提供了搜索功能，包括Ads、搜索、Homefeed、Related Pins、Visual和Shopping。 搜索系统的关键指标之一是索引延迟，即更新搜索索引以反映变化所需的时间。随着系统能力的不断增强和新用例的上线，即时索引新文档的能力变得更加重要。Manas已经支持增量索引，能够提供数十分钟以内的索引延迟。不幸的是，这不能满足日益增长的来自Ads和follow feeds的业务需求。于是在Manas中构建了一个新的模块，进一步将索引延迟降低到几分之一秒。 在这篇文章中描述了系统的架构及其关键挑战，并给出了权衡的细节。

Challenges naturally occur with Pinterest’s rapid growth. As a Pinner, you might have noticed some instances where your data doesn’t look “correct,” and you may have had a negative experience because of it. For example: the “Pin count” in your profile shows the incorrect number of Pins.

Demand on Pinterest’s core infrastructure systems is accelerating faster than ever as more Pinners come to Pinterest to find inspiration. A distributed cache layer fronting many services and databases is one of our core storage systems that sits near the bottom of Pinterest’s infrastructure stack, responsible for absorbing the vast majority of backend traffic driven by this growth.

People have always come to Pinterest for shopping inspiration, and we’ve made big strides over the years to make that as seamless as possible so Pinners (users) can go from inspiration to purchase, including evolving shoppable Product Pins, improving recommendations and making it easier for merchants to upload their catalogs to curate and feature their products.

Ankit Patel | Software engineer, Content Acquisition and Media Platform

Philip Apps | Data Scientist, Ads Quality

How experimentation and cross-functional collaboration are key to making a redesign successful

Urvashi Reddy | Software Engineer, Engineering Productivity Team Adam Berry | Tech Lead, Engineering Productivity Team Rui Li | Software…

James Ouhyoung | Growth Search Traffic, Bruce Yu | Growth Search Traffic

Rui Huang, Song Cui | Software Engineers, Content Interest Understanding Team, Jennifer Zhao | Software Engineers, Content Core Signal…

Nishant Roy | Ads Serving

Felix Zhou | Shopping, Weiran Li | Shopping, Somnath Banerjee | Shopping

The Pinterest Growth team has over 100 members, and we’ve run thousands of experiments over the years. It’s difficult to run that many experiments and still maintain a high success rate over time. We’ve found the traditional growth team model of team leads deciding which opportunities to try didn’t scale well as our team grew. Increasing the number of high-quality ideas ready for experimentation is one of the biggest levers for increasing the impact of a growth team, but our leads have less and less time to

四年了，从单一的推荐算法到多种推荐算法合并；从一个feature排序到几百维，上千维feature用机器排序；从2~3人的小项目到十几个人的团队，一起来看看Pinterest推荐系统都有哪些进化吧。