扩展Shopify的BFCM实时地图。构建一个新的Apache Flink

By Berkay Antmen, Chris Wu, and Dave Sugden

作者：Berkay Antmen, Chris Wu, 和 Dave Sugden

In 2017, various teams at Shopify came together to build an external-facing live-streamed visualization of all the sales made by Shopify merchants during the Black Friday and Cyber Monday (BFCM) weekend. We call it the Shopify BFCM live map.

2017年，Shopify的各个团队共同建立了一个面向外部的可视化直播，展示Shopify商家在黑色星期五和网络星期一（BFCM）周末的所有销售情况。我们称它为Shopify BFCM直播地图。

Shopify’s BFCM live map is a visual signal of the shift in consumer spending towards independent businesses and our way to celebrate the power of entrepreneurship. Over the years, it’s become a tradition for different teams within Shopify to iterate on the live map to see how we can better tell this story. Because of our efforts, people all over the world can watch our merchant sales in real-time, online, broadcast on television, and even in Times Square.

Shopify的BFCM实时地图是消费者消费向独立企业转变的一个视觉信号，也是我们庆祝创业力量的方式。多年来，Shopify内部不同的团队对实时地图进行迭代，看我们如何更好地讲述这个故事，这已经成为一个传统。因为我们的努力，全世界的人都可以在网上、电视上播放、甚至在时代广场上实时观看我们的商家销售。

This year, the Shopify Data Platform Engineering team played a significant role in the latest iteration of the BFCM live map. Firstly, we sought to explore what new insights we could introduce and display on the live map. Secondly, and most importantly, we needed to figure out a way to scale the live map. Last year we had more than 1 million merchants. That number has grown to over 1.7 million. With just weeks left until BFCM, we were tasked with not only figuring out how to address the system’s scalability issues but also challenging ourselves to do so in a way that would help us create patterns we could repeat elsewhere in Shopify.

今年，Shopify数据平台工程团队在BFCM实时地图的最新迭代中发挥了重要作用。首先，我们试图探索我们可以在实时地图上引入和显示哪些新的洞察力。其次，最重要的是，我们需要找出一种方法来扩展实时地图。去年，我们有超过100万的商户。这个数字已经增长到170多万。在距离BFCM仅剩几周的时候，我们的任务是不仅要想办法解决系统的可扩展性问题，还要挑战自己，以帮助我们创造可以在Shopify其他地方重复的模式。

We’ll dive into how our team, along with many others, revamped the data infrastructure powering our BFCM live map using Apache Flink. In a matter of weeks, we create...