利用Go Worker Pools来扩展服务器端的数据共享

Maintaining a service that processes over a billion events per day poses many learning opportunities. In an ecommerce landscape, our service needs to be ready to handle global traffic, flash sales, holiday seasons, failovers and the like. Within this blog, I'll detail how I scaled our Server Pixels service to increase its event processing performance by 170%, from 7.75 thousand events per second per pod to 21 thousand events per second per pod.

维护一个每天处理超过10亿个事件的服务带来了许多学习机会。在电子商务领域，我们的服务需要准备好处理全球流量、闪电销售、假日季节、故障转移等问题。在这篇博客中，我将详细介绍我是如何扩展我们的Server Pixels服务，使其事件处理性能提高170%，从每荚每秒7.75万个事件提高到每荚每秒2.1万个事件。

Capturing a Shopify customer’s journey is critical to contributing insights into marketing efforts of Shopify merchants. Similar to Web Pixels, Server Pixels grants Shopify merchants the freedom to activate and understand their customers’ behavioral data by sending this structured data from storefronts to marketing partners. However, unlike the Web Pixel service, Server Pixels sends these events through the server rather than client-side. This server-side data sharing is proven to be more reliable allowing for better control and observability of outgoing data to our partners’ servers. The merchant benefits from this as they are able to drive more sales at a lower cost of acquisition (CAC). With regional opt-out privacy regulations built into our service, only customers who have allowed tracking will have their events processed and sent to partners.  Key events in a customer’s journey on a storefront are captured such as checkout completion, search submissions and product views. Server Pixels is a service written in Golang which validates, processes, augments, and consequently, produces more than one billion customer events per day. However, with the management of such a large number of events, problems of scale start to emerge.

捕捉Shopify客户的旅程对于为Shopify商家的营销工作提供洞察力至关重要。与Web Pixels类似，Server Pixels赋予Shopify商家自由，通过从店面向营销伙伴发送这些结构化数据来激活和了解客户的行为数据。然而，与Web Pixel服务不同，Server Pixels通过服务器而不是客户端发送这些事件。这种服务器端的数据共享被证明是更可靠的，可以更好地控制和观察发往...