Containerizing Apache Hadoop Infrastructure at Uber

Containerizing Apache Hadoop Infrastructure at Uber

摘要

In 2019, we started a journey to re-architect the Hadoop deployment stack. Fast forward 2 years, over 60% of Hadoop runs in Docker containers, bringing major operational benefits to the team. As a result of the initiative, the team handed off many of their responsibilities to other infrastructure teams, and was able to focus more on core Hadoop development.

This article provides a summary of problems we faced, and how we solved them along the way.