提升 Magic Pocket 中的存储效率，我们的不可变 blob 存储

Magic Pocket is the core Dropbox storage system—a custom-built, exabyte-scale blob storage system designed for durability, availability, scale, and efficiency. It holds user content, which means it must be safe, fast, and cost-effective to scale with the company. For Dropbox, storage efficiency really matters. We measure it by looking at how much total disk space we use compared to how much user data we’re actually storing.

Magic Pocket 是 Dropbox 的核心存储系统——一个自定义构建的艾字节规模 blob 存储系统，专为耐久性、可用性、规模和效率而设计。它存储用户内容，因此必须安全、快速且成本效益高，以适应公司的规模增长。对于 Dropbox 来说，存储效率至关重要。我们通过比较总磁盘空间使用量与实际存储的用户数据量来衡量它。

Last year, we rolled out a new service that changed how data is placed across Magic Pocket. The change reduced write amplification for background writes, so each write triggered fewer backend storage operations. But it also had an unintended side effect: fragmentation increased, pushing storage overhead higher. Most of that growth came from a small number of severely under-filled volumes that consumed a disproportionate share of raw capacity, and our existing compaction strategy couldn’t reclaim the space quickly enough. At exabyte scale, even modest increases in overhead translate into meaningful infrastructure and capacity costs, so bringing that number back down quickly became a priority.

去年，我们推出了一项新服务，该服务改变了数据在 Magic Pocket 中的放置方式。这一变化减少了后台写入的写放大，因此每次写入触发的后端存储操作更少。但它也产生了一个意想不到的副作用：碎片化增加，导致存储开销更高。大部分增长来自少数严重填充不足的卷，这些卷消耗了不成比例的原始容量，而我们现有的 compaction 策略无法快速回收空间。在艾字节规模下，即使是开销的适度增加也会转化为有意义的 infrastructure 和容量成本，因此迅速降低这一数字成为优先事项。

In this post, we’ll walk through why overhead is particularly hard to control in an immutable blob store, how compaction works in Magic Pocket, and the multi-strategy approach we rolled out to drive overhead back down, even below our previous baseline.

在本文中，我们将介绍为什么在不可变 blob 存储中开销特别难控制，Magic Pocket 中的 compaction 如何工作，以及我们推出的多策略方法如何将开销驱动回下降，甚至低于之前的基线。

The cost of immutability

不可变性的代价

When users upload files to Dropbox, Magic Pocket breaks those files into smaller pieces called blobs and stores them across its s...