Database File Format Optimization: Per Column Dictionary

Database File Format Optimization: Per Column Dictionary

摘要

At Mixpanel, our proprietary multi-tenant columnar store database, known as ARB, serves as the backbone of our analytics platform. Following the ingestion of customer events, these data sets undergo a compaction process to create columnar files, which are then uploaded to Google Cloud Storage (GCS). In this blog post, we will delve into the recent updates we’ve implemented in this columnar file format. These updates enhance our query servers’ ability to accommodate more ARB files in their local SSDs, all without actually reducing file sizes. In some instances, these changes may even result in larger files.