确保Uber的Kafka®基础设施安全

Background

背景介绍

Uber has one of the largest deployments of Apache Kafka® in the world. It empowers a large number of real-time workflows at Uber, including pub-sub message buses for passing event data from the rider and driver apps, as well as financial transaction events between the backend services. As Kafka forms a critical component of Uber’s core workflows, it is important to secure the data being published and subscribed from the topics to maintain the integrity of the data and to provide an access control mechanism for who can publish/subscribe to a given topic.

Uber是世界上部署Apache Kafka®最多的公司之一。它为Uber的大量实时工作流程提供支持，包括用于传递骑手和司机应用程序的事件数据的pub-sub消息总线，以及后台服务之间的金融交易事件。由于Kafka是Uber核心工作流程的重要组成部分，因此必须确保从主题发布和订阅的数据安全，以保持数据的完整性，并为谁可以发布/订阅给定主题提供访问控制机制。

Kafka Security Concepts

Kafka安全概念

Before we dive into the core architecture of Kafka security, let’s go over some basic security concepts to help in understanding the design. The two important ones are authentication (authn.) and authorization (authz.). While authn. and authz. are often used interchangeably, they are distinct and typically have independent workflows. Simply put, authn. is the process of verifying who someone is, whereas authz. is the process of verifying to what specific data a user or a service has access.

在我们深入了解Kafka安全的核心架构之前，我们先来看看一些基本的安全概念，以帮助理解设计。其中两个重要的概念是认证（authn.）和授权（authz.）。虽然authn.和authz.经常被交替使用，但它们是不同的，通常有独立的工作流程。简单地说，authn.是验证某人是谁的过程，而authz.是验证用户或服务可以访问哪些特定数据的过程。

The situation is like that of an airline determining which people can board a flight. The first step is to confirm a passenger’s identity to make sure they are who they say they are. Once a passenger’s identity has been determined, the second step is verifying whether their booking is valid to board that specific flight.

这种情况就像航空公司确定哪些人可以登上航班一样。第一步是确认乘客的身份，确保他们是他们所说的人。一旦确定了乘客的身份，第二步是核实他们的预订是否有效，以登上该特定航班。

Encryption

加密

Data encryption between the clients and Kafka brokers can help prevent man-in-the-middle (MITM) attacks. Kafka supports SSL/TLS on the transport layer to enable encryptio...