认识Chrono,我们的可扩展、一致性的元数据缓存解决方案

Efficiently storing and retrieving metadata has long been a core focus for metadata teams at Dropbox, and in recent years we've made significant improvements to our foundational layer. By building our own incrementally scalable key-value storage, we’ve been able to avoid necessary doubling of our hardware fleet. As we scale, our evolution continues—but there are still challenges we cannot ignore.

高效地存储和检索元数据一直是Dropbox元数据团队的核心关注点,在最近几年中,我们在基础层面上取得了重大改进。通过构建我们自己的增量可扩展的键值存储,我们能够避免必要的硬件扩容。随着我们的规模扩大,我们的发展仍在继续,但仍然存在一些无法忽视的挑战。

At our current rate of growth, we knew there would come a time when the volume of read queries per second (QPS) could not be sustainably served by our existing underlying storage machines. Caching was the obvious solution—but not necessarily an easy one. 

根据我们目前的增长速度,我们知道会有一个时刻,读取查询每秒(QPS)的数量无法持续地由我们现有的底层存储机器提供服务。缓存是显而易见的解决方案,但并不一定容易。

Internal clients of Dropbox metadata have long relied on the assumption that data retrieved would provide a read-after-write guarantee (i.e., if I successfully write to a key, and then immediately subsequently read that key, absent other writes, I should see what I just wrote and not some stale value). If we were to weaken this guarantee, it would be impractical for the metadata team to audit and change every client team’s code. In addition, doing so could require significant client side code complication and/or re-design, reducing development velocity for product teams. 

Dropbox元数据的内部客户端长期以来一直依赖于这样一个假设,即检索到的数据将提供读写后的保证(即,如果我成功写入一个键,然后立即随后读取该键,在没有其他写入的情况下,我应该看到我刚刚写入的值,而不是一些过期的值)。如果我们削弱了这个保证,那么对于元数据团队来说,审计和更改每个客户端团队的代码将是不切实际的。此外,这样做可能需要在客户端代码中引入重大的复杂性和/或重新设计,从而降低产品团队的开发速度。

If we wanted to solve our high-volume read QPS problem while upholding our clients’ expectation of read consistency, traditional caching solutions would not work. We needed to find a scalable, consistent caching solution to solve both problems at once.

如果我们想要解决高读取QPS问题,并同时保持客户端对读取一致性的期望,传统的缓存解决方案将无法起作用。我们需要找到一个可扩展、一致的缓存解决方案来同时解决这两个问题。

Our answer is Chrono, a scalable, consistent caching system built ...

开通本站会员,查看完整译文。

Главная - Вики-сайт
Copyright © 2011-2024 iteam. Current version is 2.131.0. UTC+08:00, 2024-09-17 01:47
浙ICP备14020137号-1 $Гость$