使用 RocksDB 构建 Pinterest 的新宽列数据库

Rajath Prasad, Senior Engineering Manager

Rajath Prasad，高级工程经理

Pinterest serves more than 480M monthly users and has grown to be a global destination for visual inspiration. As Pinterest has grown, so have our storage requirements. In 2020, anticipating the growing needs of the business and to simplify our storage offerings, we decided to consolidate our different key-value systems in the company into a single unified service called KVStore. While KVStore was the client facing abstraction, we also built a storage service called Rockstorewidecolumn: a wide column, schemaless NoSQL database built using RocksDB. This blog post goes into the details of how we built this massively scalable, highly available wide column database using RocksDB, and provides information about the data model, APIs, and key features. Additionally, the last section explains how this new database supports a key platform in the product.

Pinterest每月服务超过4.8亿用户，已经成为全球视觉灵感的目的地。随着Pinterest的发展，我们的存储需求也在增长。2020年，为了满足业务的不断增长需求并简化我们的存储方案，我们决定将公司中不同的键值系统整合到一个名为KVStore的统一服务中。虽然KVStore是面向客户的抽象层，但我们还构建了一个名为Rockstorewidecolumn的存储服务：一个使用RocksDB构建的宽列、无模式NoSQL数据库。本博文详细介绍了我们如何使用RocksDB构建这个高度可扩展、高可用的宽列数据库，并提供了有关数据模型、API和关键特性的信息。此外，最后一节解释了这个新数据库如何支持产品中的一个关键平台。

Wide Column Databases

宽列数据库

Before we dive into the details, let’s first understand what a wide column database is. While a simple key value database can be viewed as a persistent hash map, a wide column database can be interpreted as a two dimensional key-value store with a flexible columnar structure. The key difference compared to a relational database is that the columns can vary from row to row, without a fixed schema. The diagram below shows the difference between a simple key value structure and a wide column structure.

在深入了解细节之前，让我们首先了解什么是宽列数据库。简单的键值数据库可以被视为持久的哈希映射，而宽列数据库可以被解释为具有灵活列式结构的二维键值存储。与关系数据库相比，其关键区别在于列可以在行与行之间变化，没有固定的模式。下图显示了简单键值结构和宽列结构之间的区别。

Figure 1: Difference between a simple key value database and a wide column database

图1：简单键值数据库和宽列数据库之间的区别

RocksDB

RocksDB is an open-source embe...