Netflix 如何使用 Kueue 简化批处理计算

[

Netflix TechBlog

Netflix 技术博客

](https://netflixtechblog.com/?source=post_page---publication_nav-2615bd06b42e-87860682629c---------------------------------------)

[

Netflix TechBlog

](https://netflixtechblog.com/?source=post_page---post_publication_sidebar-2615bd06b42e-87860682629c---------------------------------------)

Learn about Netflix’s world class engineering efforts, company culture, product developments and more.

了解 Netflix 世界级的工程实践、企业文化、产品开发及更多内容。

By Alvin Bao, Alex Petrov, Jennifer Lai, Aidan Sherr, and Samartha Chandrashekar

作者：Alvin Bao、Alex Petrov、Jennifer Lai、Aidan Sherr 和 Samartha Chandrashekar

As a part of the journey to transition Netflix’s compute infrastructure to be more Kubernetes-native, we have leaned into incorporating components from the Kubernetes ecosystem into our container platform Titus. One example of this is our use of Kueue, a cloud-native job queueing system for batch workloads, which has largely replaced the custom queuing and scheduling logic in our homegrown managed batch solution Compute Managed Batch (CMB). In this post, we’ll give an overview of what motivated the migration, how we migrated millions of batch jobs to use Kueue, and what Kueue allows us to offer as a Compute platform.

作为将 Netflix 计算基础设施向更 Kubernetes 原生方向转型之旅的一部分，我们致力于将 Kubernetes 生态系统中的组件整合到我们的容器平台 Titus 中。其中一个例子是我们使用了 Kueue，这是一个用于批处理工作负载的云原生作业排队系统，它在很大程度上取代了我们自研的托管批处理解决方案 Compute Managed Batch (CMB) 中的自定义排队和调度逻辑。在本文中，我们将概述促使此次迁移的原因、我们如何将数百万个批处理作业迁移到使用 Kueue，以及 Kueue 作为计算平台能让我们提供哪些功能。

Brief Overview of CMB and Titus

CMB 和 Titus 简介

CMB is a managed batch solution that allows users and applications to execute and manage workloads that run to completion. Using a tenant hierarchy, workloads are managed and queued with ordered execution through prior...