在Java中处理不稳定的单元测试

Unit testing forms the bedrock of any Continuous Integration (CI) system. It warns software engineers of bugs in newly-implemented code and regressions in existing code, before it is merged. This ensures increased software reliability. It also improves overall developer productivity, as bugs are caught early in the software development lifecycle. Hence, building a stable and reliable testing system is often a key requirement for software development organizations.

单元测试是任何持续集成（CI）系统的基石。在合并之前，它警告软件工程师新实施的代码中存在的错误和现有代码中的回归。这确保了软件可靠性的提高。它也提高了整个开发人员的生产力，因为在软件开发生命周期的早期就能发现错误。因此，建立一个稳定和可靠的测试系统往往是软件开发组织的一个关键要求。

Unfortunately, by definition, flaky unit tests undercut this requirement. A unit test is considered flaky if it returns different results (pass or fail) on any two executions, without any underlying changes to the source code. A flaky test can occur either due to program-level non-determinism (e.g., thread ordering and other concurrency issues) within the test code or the code being tested. Alternatively, it can occur due to variability in the testing environment (e.g., the machine on which it is executed, the set of tests that are executed concurrently, etc.). While the former requires fixing the code, the latter involves identifying the reasons that resulted in the non-determinism, and addressing them to remove the flakiness. The testing of both code patterns and infrastructure must be geared towards diminishing the potential for flaky tests to arise.

不幸的是，从定义上看，不稳定的单元测试破坏了这个要求。如果一个单元测试在任何两次执行中返回不同的结果（通过或失败），而没有对源代码进行任何基本的修改，那么它就被认为是不稳定的。一个不稳定的测试可能是由于测试代码或被测代码中的程序级非确定性（例如，线程排序和其他并发性问题）。或者，它可能是由于测试环境的可变性（例如，执行的机器，并发执行的测试集，等等）。前者需要修复代码，而后者则需要确定导致非确定性的原因，并解决这些原因以消除飘忽不定的现象。对代码模式和基础设施的测试必须着眼于减少易变性测试产生的可能性。

Flaky tests affect developer productivity across multiple dimensions. First, when a test fails due to extraneous reasons, the underlying issue has to be investigated, which can be time-consuming, given the non-deterministic reproducibility of the failure. In many cases, reproducing the failure locally may be impr...