在没有停机的情况下大规模迁移关键流量 - 第一部分

Shyam Gala, Javier Fernandez-Ivern, Anup Rokkam Pratap, Devang Shah

Hundreds of millions of customers tune into Netflix every day, expecting an uninterrupted and immersive streaming experience. Behind the scenes, a myriad of systems and services are involved in orchestrating the product experience. These backend systems are consistently being evolved and optimized to meet and exceed customer and product expectations.

数以亿计的客户每天都在收听Netflix，期待着不间断的、身临其境的流媒体体验。在幕后，无数的系统和服务参与了产品体验的协调工作。这些后端系统一直在不断发展和优化，以满足并超越客户和产品的期望。

When undertaking system migrations, one of the main challenges is establishing confidence and seamlessly transitioning the traffic to the upgraded architecture without adversely impacting the customer experience. This blog series will examine the tools, techniques, and strategies we have utilized to achieve this goal.

在进行系统迁移时，主要的挑战之一是建立信心，将流量无缝过渡到升级后的架构，而不对客户体验产生不利影响。本博客系列将研究我们为实现这一目标所使用的工具、技术和策略。

The backend for the streaming product utilizes a highly distributed microservices architecture; hence these migrations also happen at different points of the service call graph. It can happen on an edge API system servicing customer devices, between the edge and mid-tier services, or from mid-tiers to data stores. Another relevant factor is that the migration could be happening on APIs that are stateless and idempotent, or it could be happening on stateful APIs.

流媒体产品的后端采用了高度分布式的微服务架构；因此这些迁移也发生在服务调用图的不同点。它可以发生在为客户设备服务的边缘API系统上，也可以发生在边缘和中层服务之间，或者从中层到数据存储。另一个相关因素是，迁移可能发生在无状态和空闲的API上，也可能发生在有状态API上。

We have categorized the tools and techniques we have used to facilitate these migrations in two high-level phases. The first phase involves validating functional correctness, scalability, and performance concerns and ensuring the new systems’ resilience before the migration. The second phase involves migrating the traffic over to the new systems in a manner that mitigates the risk of incidents while continually monitoring and confirming that we are meeting crucial metrics tracked a...