DoorDash 通过 Kafka 自助服务为工程师提供支持

DoorDash is supporting an increasingly diverse array of infrastructure use cases as the company matures. To maintain our development velocity and meet growing demands, we are transitioning toward making our stateful storage offerings more self-serve. This journey began with Kafka, one of our most critical and widely used infrastructure components.

随着公司的成熟，DoorDash正在支持越来越多样化的基础设施用例。为了保持我们的开发速度并满足不断增长的需求，我们正在向使我们的有状态存储提供更多自助服务的方向过渡。这个旅程始于Kafka，它是我们最关键和广泛使用的基础设施组件之一。

Kafka is a distributed event streaming platform that DoorDash uses to handle billions of real-time events. Our engineers — the primary users of this platform — manage Kafka-related resources such as topics, users, and access control lists, or ACLs, through Terraform. However, managing these resources manually was cumbersome and created a significant support burden for our infrastructure engineers because of the need to review and approve multiple pull requests.

Kafka是一个分布式事件流平台，DoorDash使用它来处理数十亿个实时事件。我们的工程师——这个平台的主要用户——通过Terraform管理与Kafka相关的资源，如主题、用户和访问控制列表（ACL）。然而，手动管理这些资源很繁琐，给我们的基础设施工程师带来了重大的支持负担，因为需要审查和批准多个拉取请求。

To address this, we developed Kafka Self-Serve, our flagship self-serve storage infrastructure platform. Kafka Self-Serve empowers product engineers to onboard and manage their Kafka resources with minimal interaction from infrastructure engineers. This system was built in response to feedback from our product engineers and a careful examination of the support burden on our infrastructure team.

为了解决这个问题，我们开发了Kafka Self-Serve，我们的旗舰自助存储基础设施平台。Kafka Self-Serve使产品工程师能够在最小程度与基础设施工程师进行交互的情况下入门和管理他们的Kafka资源。该系统是根据我们的产品工程师的反馈和对基础设施团队的支持负担进行仔细考察而构建的。

Here we share our Kafka Self-Serve platform development journey, the challenges we faced along the way, and the solutions we implemented. We hope to provide valuable insights that can help the broader tech community embark on similar journeys toward self-serve infrastructure.

在这里，我们分享了我们Kafka自助平台的开发历程，我们在这个过程中面临的挑战以及我们实施的解决方案。我们希望能够提供有价值的见解，帮助更广泛的技术社区踏上类似的自助基础设施之旅。

Before we implemented the Kafka Self-Serv...