优化我们的E2E管道
In the world of DevOps and Developer Experience (DevXP), speed and efficiency can make a big difference on an engineer’s day-to-day tasks. Today, we’ll dive into how Slack’s DevXP team took some existing tools and used them to optimize an end-to-end (E2E) testing pipeline. This lowered build times and reduced redundant processes, saving both time and resources for engineers at Slack.
在DevOps和开发者体验(DevXP)的世界中,速度和效率可以对工程师的日常任务产生重大影响。今天,我们将深入探讨Slack的DevXP团队如何利用一些现有工具来优化端到端(E2E)测试管道。这降低了构建时间,减少了冗余流程,为Slack的工程师节省了时间和资源。
The Problem: Unnecessary Frontend Builds
问题: 不必要的前端构建
For one of our largest code repositories (a monolithic repository, or monorepo), Slack has a CI/CD pipeline that runs E2E tests before merging code into the main
branch. This is critical for ensuring that changes are validated across the entire stack for the Slack application: frontend, backend, database, and the handful of services in between. However, we noticed a bottleneck: building the frontend code was taking longer than expected and occurred too frequently, even when there were no frontend-related changes. Here’s the breakdown:
在我们最大的代码库之一(一个单体代码库,或称monorepo)中,Slack有一个CI/CD管道,在将代码合并到main
分支之前运行E2E测试。这对于确保Slack应用程序的整个堆栈(前端、后端、数据库以及其中的少数服务)中的更改得到验证至关重要。然而,我们注意到一个瓶颈:构建前端代码所需的时间比预期的要长,并且即使没有前端相关的更改,这种情况也发生得太频繁。以下是详细情况:
- Developer Workflow: A developer makes changes and pushes to a branch.
- 开发者工作流程: 开发者进行更改并推送到一个分支。
- Build Process: The frontend code is built (~5 minutes).
- 构建过程: 前端代码构建 (~5 分钟)。
- Deployment: The build is deployed to a QA environment.
- 部署: 构建被部署到 QA 环境。
- Testing: We run over 200 E2E tests, taking another 5 minutes.
- 测试: 我们运行超过 200 个 E2E 测试,耗时另需 5 分钟。
This entire process took about 10 minutes per run. Half of that time, around 5 minutes, was consumed by frontend builds, even when no frontend changes were involved.
整个过程每次运行大约耗时10分钟。其中一半时间,即约5分钟,被前端构建消耗,即使没有涉及前端更改。
Given that hundreds of pull requests (PRs) are merged daily, these redundant builds were not only time-consuming, but costly:
考虑到每天有数百个拉取请求(PR)被合并,这些冗余构建不仅耗时,而且成本高昂:
- Thousands...