在Pinterest的统一PubSub客户端

Vahid Hashemian | Software Engineer, Logging Platform
Jeff Xiang | Software Engineer, Logging Platform

Vahid Hashemian | 软件工程师，记录平台
Jeff Xiang | 软件工程师，记录平台

At Pinterest, the Logging Platform team manages the PubSub layer and provides support for clients that interact with it. At the heart of the PubSub layer, there are two main systems responsible for ingress and egress of data in motion:

在Pinterest，日志平台团队负责管理PubSub层，并为与之互动的客户端提供支持。在PubSub层的核心，有两个主要系统负责运动中数据的进入和退出。

Apache Kafka® (Kafka hereafter)
Apache Kafka®(以下简称Kafka)
MemQ
姆姆Q

Over the last several years, we have learnt through operational experience that our customers and business needs to have reduced KTLO costs, and they want the platform team to own not just the servers / service but also the client / SDK as well as the on-call associated with any issues that arise from client-server connectivity so they can focus on application logic.

在过去的几年里，我们通过运营经验了解到，我们的客户和企业需要降低KTLO成本，他们希望平台团队不仅拥有服务器/服务，还拥有客户端/SDK，以及与客户端-服务器连接所产生的任何问题有关的待命，这样他们就可以专注于应用逻辑。

As a platform team, we have a desire to improve the scalability and efficiency of our platform (e.g. by improving PubSub systems) which in turn may require making rapid changes to the client / SDK.

作为一个平台团队，我们希望提高我们平台的可扩展性和效率（例如通过改进PubSub系统），这反过来可能需要对客户端/SDK进行快速修改。

In order to accomplish our objective of improving the Quality of Service (QoS) for the Logging Platform we decided to invest in and take ownership of PubSub clients / SDK. Using the native clients and building on top of them to add several value-add features, albeit a huge undertaking, would help solve our scalability, stability and dev velocity objectives. That is why we decided to implement a unified PubSub client library called PubSub Client (PSC).

为了实现我们提高日志平台服务质量（QoS）的目标，我们决定投资并拥有PubSub客户端/SDK。使用本地客户端并在其基础上添加一些增值功能，尽管是一个巨大的工程，但将有助于解决我们的可扩展性、稳定性和开发速度的目标。这就是为什么我们决定实施一个统一的PubSub客户端库，称为PubSub客户端（PSC）。

Why Another Client?

为什么是另一个客户？

Until about two years ago Kafka was the single PubSub system at Pinterest, and the majority of data pipelines still run on...