通过VPC端点服务暴露一个Kafka集群

In large organisations, it is a common practice to isolate the cloud resources of different verticals. Amazon Web Services (AWS) Virtual Private Cloud (VPC) is a convenient way of doing so. At Grab, while our core AWS services reside in a main VPC, a number of Grab Tech Families (TFs) have their own dedicated VPC. One such example is GrabKios. Previously known as “Kudo”, GrabKios was acquired by Grab in 2017 and has always been residing in its own AWS account and dedicated VPC.

在大型组织中，隔离不同垂直领域的云资源是一种常见的做法。亚马逊网络服务（AWS）的虚拟私有云（VPC）是一种方便的方式。在 Grab，虽然我们的核心 AWS 服务驻扎在一个主要的 VPC 中，但一些 Grab Tech Families (TFs) 有自己的专用 VPC。GrabKios 就是这样一个例子。GrabKios 以前被称为 "Kudo"，于 2017 年被 Grab 收购，并一直驻留在自己的 AWS 账户和专用 VPC 中。

In this article, we explore how we exposed an Apache Kafka cluster across multiple Availability Zones (AZs) in Grab’s main VPC, to producers and consumers residing in the GrabKios VPC, via a VPC Endpoint Service. This design is part of Coban unified stream processing platform at Grab.

在这篇文章中，我们将探讨如何通过VPC端点服务，将Grab主VPC中多个可用区（AZ）的Apache Kafka集群暴露给GrabKios VPC中的生产者和消费者。这个设计是 Grab 的 Coban 统一流处理平台的一部分。

There are several ways of enabling communication between applications across distinct VPCs; VPC peering is the most straightforward and affordable option. However, it potentially exposes the entire VPC networks to each other, needlessly increasing the attack surface.

有几种方法可以实现不同VPC之间的应用通信；VPC对接是最直接和最实惠的选择。然而，它有可能使整个VPC网络相互暴露，不必要地增加攻击面。

Security has always been one of Grab’s top concerns and with Grab’s increasing growth, there is a need to deprecate VPC peering and shift to a method of only exposing services that require remote access. The AWS VPC Endpoint Service allows us to do exactly that for TCP/IPv4 communications within a single AWS region.

安全性一直是 Grab 最关心的问题之一，随着 Grab 的不断发展，有必要废除 VPC 对等，转而采用只暴露需要远程访问的服务的方法。AWS VPC 端点服务使我们能够在单个AWS 区域内的 TCP/IPv4 通信中做到这一点。

Setting up a VPC Endpoint Service compared to VPC peering is already relatively complex. On top of that, we need to expose an Apache Kafka cluster via such an endpoint, whi...