使用Google Cloud Platform现代化Uber的批处理数据基础设施

Uber runs one of the largest Hadoop installations in the world. Our Hadoop ecosystem hosts more than 1 exabyte of data across tens of thousands of servers in each of our two regions. The open source data ecosystem, including the Hadoop ecosystem discussed in previous engineering blogs, has been the core of our data platform.

Uber 是全球最大的 Hadoop 部署之一。我们的Hadoop 生态系统在每个区域的数万台服务器上托管了超过 1EB 的数据。开源数据生态系统，包括之前工程博客中讨论的 Hadoop 生态系统，一直是我们数据平台的核心。

Over the past few months, we have been assessing our platform and infrastructure needs to make sure we are well positioned to modernize our big data infrastructure to keep up with the growing needs of Uber.

在过去的几个月中，我们一直在评估我们的平台和基础设施需求，以确保我们能够与Uber不断增长的需求保持现代化的大数据基础设施。

Today, we are excited to announce that we are working with Google Cloud Platform (GCP) to move our batch data analytics and ML training stack to GCP. 

今天，我们很高兴宣布我们正在与Google Cloud Platform（GCP）合作，将我们的批处理数据分析和ML训练堆栈迁移到GCP。

Uber Data Platform’s mission is to democratize data-driven business decisions through intuitive, reliable, and efficient data products. Modernizing with GCP will enable big gains in user productivity, engineering velocity, improved cost efficiency, access to new innovation, and expanded data governance.

Uber数据平台的使命是通过直观、可靠和高效的数据产品推动数据驱动的业务决策。通过与GCP的现代化，将实现用户生产力的大幅提升、工程速度的加快、成本效率的改善、获得新的创新和扩展数据治理。

Our strategy for the initial migration to GCP is to leverage cloud’s object store for the data lake storage while migrating the rest of the data stack to cloud IaaS (Infrastructure as a Service). This approach facilitates a fast migration path with minimum disruption to existing jobs and pipelines as we can replicate the exact versions of our on-prem software stack, engines, and security model on IaaS. We plan to adopt applicable PaaS (Platform as a Service) offerings, for example GCP Dataproc or BigQuery, after the initial migration to GCP to take full advantage of the elasticity and performance benefits cloud native services provide. Our plan is to execute on this strategy over the next several quarters, do...