Spinner:大规模迁移至Pinterest的新工作流程平台

This article is a repost from the author’s original account here.

本文转自作者的原始账户 这里.

In our last blog post, we discussed how we made the decision and took the actions to move from our legacy system, Pinball, to our new system, Spinner, which is built on top of the Apache Airflow project. As a reminder, this is based off of a custom branch that branched off of Airflow version 1.10-stable with some features cherry picked from the master branch.

在上一篇博文中，我们讨论了我们是如何做出决定并采取行动，从我们的传统系统Pinball转移到我们的新系统Spinner，它是建立在Apache Airflow项目之上的。作为提醒，这是基于一个自定义分支，该分支从Airflow 1.10-stable版本中分离出来，并从主分支中挑出一些功能。

In this post, we will explain how we approached and designed the migration, identified requirements, and coordinated with all our engineer teams to seamlessly migrate 3000+ workflows to Airflow. We will deep dive into trade offs made, but before we do that, we want to give our learnings.

在这篇文章中，我们将解释我们如何对待和设计迁移，确定需求，并与我们所有的工程师团队协调，将3000多个工作流程无缝迁移到Airflow。我们将深入探讨所做的交易，但在这之前，我们想给出我们的经验。

The keys to a successful migration process by our standards were:

按照我们的标准，成功迁移过程的关键是。

Understand and fulfill the gaps between Airflow and the in-house workflow system we previously had. We identify the feature gap, the security differences, and the terminology that users are accustomed to.
了解并实现Airflow和我们之前的内部工作流系统之间的差距。我们找出功能上的差距，安全上的差异，以及用户所习惯的术语。
Provide migration tooling to have a low-cost way to mass migrate many workflows simultaneously and have a way for verification.
提供迁移工具，以低成本的方式同时大规模迁移许多工作流程，并有办法进行验证。
Have clear and constant user communication and onboarding materials such as wikis, tra...