eBay如何为批量应用升级开发自动测试解决方案

This article introduces a new testing methodology for 500+ batch applications at eBay. These batch applications contain 10,000+ jobs running everyday to handle offline eBay business, for example, the monthly invoice jobs are responsible to record all transactions on a customer's primary account for subscriptions. In this piece, we explore the challenges of batch application testing when upgrading — and provide a solution to resolve the pain points of testing.

本文介绍了eBay公司500多个批处理应用程序的新测试方法。这些批处理应用程序包含每天运行的10000多个作业，以处理eBay的离线业务，例如，每月的发票作业负责记录客户订阅的主要账户上的所有交易。在这篇文章中，我们探讨了升级时批量应用测试的挑战--并提供了解决测试痛点的方案。

A batch application is different from the service system, because it has no web container inside an instance. For a batch pool, it will wait for the job command from an upstream job scheduler to run based on a given schedule. Once the job command is complete, the application process will quit, and return the exit code and exit state back to the scheduler.

批量应用与服务系统不同，因为它在一个实例里面没有Web容器。对于一个批处理池来说，它将等待来自上游工作调度器的工作命令，根据给定的时间表运行。一旦作业命令完成，应用程序进程将退出，并将退出代码和退出状态返回给调度器。

Batch testing is always a challenge. Generally, a batch application is not idempotent since most are manipulating files, reading databases, etc. Those data sources can be updated frequently, and a batch job can’t get the same results even with the same input parameters.

批量测试总是一个挑战。一般来说，批处理程序不是空闲的，因为大多数是操作文件、读取数据库等。这些数据源可能会经常更新，即使输入参数相同，批处理作业也不可能得到相同的结果。

The traditional way of testing new changes, such as a framework upgrade, for a batch application is to deploy the new code directly into production for an online verification. This can be a risky approach, and there should be better testing to cover the below use cases.

测试新变化的传统方法，如框架升级，对于批处理应用程序来说，是将新代码直接部署到生产中进行在线验证。这可能是一个有风险的方法，应该有更好的测试来覆盖下面的用例。

Testing jobs that can only run once.
测试工作只能运行一次。
Some batch applications do not have enough end-to-end test cases to support testing. Thus, there is no baseline for existing behavior that can validate behavior on the new platform.
一些批量应用没有足够的端到端测试案例来支持测试。因此，没有现有行为的基...