大幅降低Uber大数据成本的挑战和机遇

Big data is at the core of Uber’s business. We continue to innovate and provide better experiences for our earners, riders, and eaters by leveraging big data, machine learning, and artificial intelligence technology. As a result, over the last four years, the scale of our big data platform multiplied from single-digit petabytes to many hundreds of petabytes.

大数据是Uber业务的核心。我们不断创新，通过利用大数据、机器学习和人工智能技术，为我们的挣钱者、乘客和吃饭者提供更好的体验。因此，在过去的四年里，我们的大数据平台的规模从个位数的PB倍增到许多数百PB。

Uber’s big data stack is built on top of the open source ecosystem. We run some of the largest deployments of Hadoop, Hive, Spark, Kafka, Presto, and Flink in the world. Open source software allows us to quickly scale up to meet Uber’s business needs without reinventing the wheel.

Uber的大数据栈是建立在开源生态系统之上的。我们运行着世界上最大的一些Hadoop、Hive、Spark、Kafka、Presto和Flink的部署。开源软件使我们能够迅速扩大规模，以满足Uber的业务需求，而无需重新发明车轮。

The cost of running our big data platform also rose significantly in that same period. The Big Data Platform was one of the most costly among the 3 internal platforms at Uber. That was when we started taking a serious look at our big data platform’s cost, aiming to reduce overhead while preserving the reliability, productivity and the value it provides to the business.

在同一时期，我们的大数据平台的运行成本也大幅上升。大数据平台是Uber的3个内部平台中成本最高的一个。就在那时，我们开始认真审视大数据平台的成本，旨在减少开销，同时保留其可靠性、生产力和为企业提供的价值。

At the end of 2019, we exceeded our goal by saving the company many millions of dollars worth of hardware. In early 2020, we developed a roadmap for the next 2 years, with the goal to cut down our big data cost even more. Over the following three posts we will outline how we successfully brought down Uber’s big data overhead from a top-level perspective, illustrate particular examples of what worked, and lay out plans for future development and research in this area.

在2019年底，我们超过了我们的目标，为公司节省了价值数百万美元的硬件。在2020年初，我们制定了未来2年的路线图，目标是进一步削减我们的大数据成本。在接下来的三篇文章中，我们将从顶层的角度概述我们是如何成功地降低Uber的大数据开销的，说明特定的成功案例，并为这个领域的未来发展和研究制定计划。

Motivation and Background

动机和背景

Uber’s Exponential Business Growth

Uber的指数式业务增长

Uber started...