Introducing Shadower: A Minimalistic Load Testing Tool

Introducing Shadower: A Minimalistic Load Testing Tool

摘要

Shadower is a load testing tool that allows us to provide load testing as a service to any microservice at Uber.

Shadower started as a command line application that allowed us to read a local file to load test a local application. At the time, Maps PEs were heavily investing in Java GC tuning. We needed Shadower to be able to do request mirroring to make sure two different applications get about the same load and different types of loads (test multiple endpoints). In summary, we were starting the same application twice: once with the production configuration and once with the new GC tuning. How’s this different from testing on staging? There are a few differences:

• Velocity: Testing on staging means pushing code to a branch, generating a build, and deploying it. This can take ~10 minutes compared to seconds locally, because locally we don’t need to generate a new build for configuration changes.
• Complexity: Our current infrastructure requires extra steps to allow multiple builds to run concurrently in the same environment, the reason for this is to reduce inconsistencies.
• Reliability: The current tools didn’t provide request mirroring, so we are at the mercy of the load balancing algorithm and how well randomized the requests are that we are using for our load test.
• Ease of use: The current tools needed developers to write code to be able to onboard a new service/endpoint.

Shadower stayed as a command line application for about a year until we built Ballast. Ballast required a reliable and easy-to-manage load test generator. Ballast is in charge of tracking resources utilization and deciding what is the right load for any load test. Shadower only focuses on executing those load tests.