在 Pinterest 生产中运行统一 PubSub 客户端
Jeff Xiang | Software Engineer, Logging Platform
Jeff Xiang | 软件工程师，日志平台
Vahid Hashemian | Software Engineer, Logging Platform
Vahid Hashemian | 软件工程师，日志平台
Jesus Zuniga | Software Engineer, Logging Platform
Jesus Zuniga | 软件工程师，日志平台
At Pinterest, data is ingested and transported at petabyte scale every day, bringing inspiration for our users to create a life they love. A central component of data ingestion infrastructure at Pinterest is our PubSub stack, and the Logging Platform team currently runs deployments of Apache Kafka and MemQ. Over the years, operational experience has taught us that our customers and business would greatly benefit from a unified PubSub interface that the platform team owns and maintains, so that application developers can focus on application logic instead of spending precious hours debugging client-server connectivity issues. Value-add features on top of the native clients can also help us achieve more ambitious goals for dev velocity, scalability, and stability. For these reasons, and others detailed in our original PubSub Client blog post, our team has decided to invest in building, productionalizing, and most recently open-sourcing PubSub Client (PSC).
在 Pinterest，每天都会以 PB 级别摄取和传输数据，为我们的用户带来创造他们喜欢的生活的灵感。Pinterest 数据摄取基础设施的一个核心组件是我们的 PubSub 堆栈，Logging Platform 团队目前运行 Apache Kafka 和 MemQ 的部署。多年的运营经验告诉我们，我们的客户和业务将极大地受益于一个统一的 PubSub 接口，由平台团队拥有和维护，这样应用程序开发人员就可以专注于应用程序逻辑，而不是花费宝贵的时间调试客户端-服务器连接问题。在本机客户端之上的增值功能也可以帮助我们实现更雄心勃勃的开发速度、可扩展性和稳定性目标。出于这些原因，以及在我们最初的 PubSub 客户端博客文章中详细说明的其他原因，我们的团队决定投资于构建、生产化和最近开源PubSub 客户端 (PSC)。
In the 1.5 years since our previous blog post, PSC has been battle-tested at large scale in Pinterest with notably positive feedback and results. From dev velocity and service stability improvements to seamless migrations from native client to PSC, we would like to share some of our findings from running a unified PubSub client library in production.
自我们上一篇博客文章以来的1.5年中，PSC在Pinterest中经过了大规模的实战测试，得到了显著的反馈和结果。从开发速度和服务稳定性的提高到从本地客户端到PSC的无缝迁移，我们想分享一些在生产中运行统一的PubSub客户端库的发现。