革命性的软件测试：推出基于LLM的bug捕捉器

Meta’s Automated Compliance Hardening (ACH) tool is a system for mutation-guided, LLM-based test generation. ACH hardens platforms against regressions by generating undetected faults (mutants) in source code that are specific to a given area of concern and using those same mutants to generate tests. When applied to privacy, for example, ACH automates the process of searching for privacy-related faults and preventing them from entering our systems in the future, ultimately hardening our code bases to reduce risk of any privacy regression.

Meta的自动合规强化（ACH）工具是一个基于变异引导的LLM测试生成系统。ACH通过在源代码中生成特定于某一关注领域的未被检测到的故障（变异体）来增强平台抵御回归的能力，并利用这些变异体生成测试。例如，在隐私方面，ACH自动化了搜索与隐私相关的故障的过程，并防止它们在未来进入我们的系统，最终增强我们的代码库以降低任何隐私回归的风险。

ACH automatically generates unit tests that target a particular kind of fault. We describe the faults we care about to ACH in plain text. The description can be incomplete, and even self-contradictory, yet ACH still generates tests that it proves will catch bugs of the kind described.

ACH自动生成针对特定类型故障的单元测试。我们以普通文本向ACH描述我们关心的故障。描述可以是不完整的，甚至是自相矛盾的，但ACH仍然生成它证明能够捕捉到所描述类型的错误的测试。

Traditionally, automated test generation techniques sought merely to increase code coverage. As every tester knows, this is only part of the solution because increasing coverage doesn’t necessarily find faults.

传统上，自动化测试生成技术仅仅寻求增加代码覆盖率。正如每个测试人员所知，这只是解决方案的一部分，因为增加覆盖率并不一定能发现缺陷。

ACH is a radical departure from this tradition, because it targets specific faults, rather than uncovered code, although it often also increases coverage in the process of targeting faults. Furthermore, because ACH is founded on the principles of Assured LLM-based Software Engineering, it keeps verifiable assurances that its tests do catch the kind of faults described.

ACH 是对这一传统的根本性突破，因为它针对特定的故障，而不是未覆盖的代码，尽管在针对故障的过程中，它通常也会增加覆盖率。此外，由于 ACH 基于有保障的基于 LLM 的软件工程原理，因此它保持可验证的保证，确保其测试确实捕捉到所描述的故障类型。

Our new research paper, “Mutation-Guided LLM-based Test Generation at Meta,” gives details of the underlying scientific foundations for ACH and how we apply ACH to privacy testing, but t...