一个优雅的平台
Coban is Grab’s real-time data streaming platform team. As a platform team, we thrive on providing our internal users from all verticals with self-served data-streaming resources, such as Kafka topics, Flink and Change Data Capture (CDC) pipelines, various kinds of Kafka-Connect connectors, as well as Apache Zeppelin notebooks, so that they can effortlessly leverage real-time data to build intelligent applications and services.
Coban是Grab的实时数据流平台团队。作为一个平台团队,我们致力于为各个垂直领域的内部用户提供自助数据流资源,例如Kafka主题,Flink和Change Data Capture(CDC)流水线,各种Kafka-Connect连接器,以及Apache Zeppelin笔记本,以便他们可以轻松利用实时数据构建智能应用和服务。
In this article, we present our journey from pure Infrastructure-as-Code (IaC) towards a more sophisticated control plane that has revolutionised the way data streaming resources are self-served at Grab. This change also leads to improved scalability, stability, security, and user adoption of our data streaming platform.
在本文中,我们介绍了我们从纯基础设施即代码(IaC)向更复杂的控制平面的转变,这一转变彻底改变了Grab自助式数据流资源的方式。这一变化还带来了我们数据流平台的可扩展性、稳定性、安全性和用户采用率的提升。
Problem statement
问题陈述
In the early ages of public cloud, it was a common practice to create virtual resources by clicking through the web console of a cloud provider, which is sometimes referred to as ClickOps.
在公共云的早期阶段,通过点击云提供商的Web控制台创建虚拟资源是一种常见做法,有时被称为ClickOps。
ClickOps has many downsides, such as:
ClickOps有很多缺点,例如:
- Inability to review, track, and audit changes to the infrastructure.
- 无法审查、跟踪和审核基础设施的变更。
- Inability to massively scale the infrastructure operations.
- 无法大规模扩展基础设施操作。
- Inconsistencies between environments, e.g. staging and production.
- 环境之间的不一致,例如分期和生产。
- Inability to quickly recover from a disaster by re-creating the infrastructure at a different location.
- 无法通过在不同位置重新创建基础架构来快速从灾难中恢复。
That said, ClickOps has one tremendous advantage; it makes creating resources using a graphical User Interface (UI) fairly easy for anyone like Infrastructure Engineers, Software Engineers, Data Engineers etc. This also leads to a high iteration speed towards innovation in general.
话虽如此,ClickOps具有一个巨大的优势;它使得使用图形用户界面(UI)创建资源对于...