Uber的基于MyRocks的分布式数据库中的差异备份

Uber uses MySQL® as the underlying storage layer for Schemaless and Docstore, Uber’s in-house distributed databases. Storing tens of petabytes of operational data and serving tens of millions of requests/second, it is one of the largest database services at Uber, used by microservices from all business verticals and serves critical Uber use cases worldwide. 

Uber将MySQL®作为底层存储层用于Schemaless和Docstore，这是Uber内部的分布式数据库。存储数十PB的运营数据并每秒提供数千万次请求，它是Uber最大的数据库服务之一，被所有业务垂直领域的微服务使用，并为全球范围内的关键Uber用例提供支持。 

The Uber Storage Platform team migrated all Schemaless and Docstore instances to MyRocks, a RocksDB storage engine based MySQL® version. The primary motivation was that RocksDB engine is optimized for write and outstanding storage space efficiency.

Uber Storage Platform团队将所有Schemaless和Docstore实例迁移到了基于RocksDB存储引擎的MyRocks，这是一个基于RocksDB的MySQL®版本。主要动机是RocksDB引擎针对写入和存储空间效率进行了优化。

In this post, we are going to talk about our journey of how we solved the subsequent issues with a lack of incremental backups for MyRocks, and the interesting technical challenges that we faced along the way.

在本文中，我们将讨论我们如何解决MyRocks缺乏增量备份的问题以及我们在解决这些问题时遇到的有趣的技术挑战。

Uber’s operation, bridging the physical and digital realms, relies heavily on robust, distributed databases, which are essential for a range of critical functions from disaster recovery to business continuity and compliance. These databases also support vital auditing and data analysis efforts across the company. 

Uber的运营依赖于强大的分布式数据库，这些数据库在从灾难恢复到业务连续性和合规性等一系列关键功能中起着重要作用。这些数据库还支持公司范围内的重要审计和数据分析工作。

Schemaless and Docstore have a layered architecture and a Docstore deployment is called an instance. In the architecture of a single Schemaless or Docstore instance, data is organized into shards. These shards are further distributed across multiple physical partitions to ensure scalability and manageability. Each partition houses redundant copies of the data, all located within a single region on individual MySQL® servers. Notably, each server is configured to store a single RocksDB column family, optimi...