DragonCrawl：用于高质量移动测试的生成式人工智能

The Developer Platform team at Uber is consistently developing new and innovative ideas to enhance the developer’s experience and strengthen the quality of our apps. Quality and testing go hand in hand, and in 2023 we took on a new and exciting challenge to change how we test our mobile applications, with a focus on machine learning (ML). Specifically, we are training models to test our applications just like real humans would.

Uber的开发者平台团队一直在不断开发新的创新想法，以提升开发者的体验和加强我们应用程序的质量。质量和测试是相辅相成的，2023年我们迎来了一个新的令人兴奋的挑战，改变了我们测试移动应用程序的方式，重点是机器学习（ML）。具体而言，我们正在训练模型以像真正的人类一样测试我们的应用程序。

Mobile testing remains an unresolved challenge, especially at our scale, encompassing thousands of developers and over 3,000 simultaneous experiments. Manual testing is usually carried out, but with high overhead, it cannot be done extensively for every minor code alteration. While test scripts can offer better scalability, they are also not immune to frequent disruptions caused by minor updates, such as new pop-ups and changes in buttons. All of these changes, no matter how minor, require recurring manual updates to the test scripts. Consequently, engineers working on this invest 30–40% of their time on maintenance. Furthermore, the substantial maintenance costs of these tests significantly hinder their adaptability and reusability across diverse cities and languages (imagine having to hire manual testers or mobile engineers for the 50+ languages that we operate in!), which makes it really difficult for us to efficiently scale testing and ensure Uber operates with high quality globally.

移动测试仍然是一个未解决的挑战，尤其是在我们的规模下，涵盖了数千名开发人员和超过3,000个同时进行的实验。通常进行手动测试，但由于开销较大，无法对每个较小的代码更改进行广泛测试。虽然测试脚本可以提供更好的可扩展性，但它们也无法免受由于较小的更新（例如新的弹出窗口和按钮更改）而引起的频繁中断的影响。所有这些更改，无论多么小，都需要定期手动更新测试脚本。因此，从事这项工作的工程师将30-40％的时间投入到维护工作中。此外，这些测试的大量维护成本严重阻碍了它们在不同城市和语言之间的适应性和可重用性（想象一下，我们必须为我们运营的50多种语言雇佣手动测试人员或移动工程师！），这使得我们难以有效地扩展测试并确保Uber在全球范围内具有高质量。

To solve these problems, we created DragonCrawl, a system that uses large language models (LLMs) to execute mobile tests with the intuition of a h...