Mobile Performance @ Lyft
In Q2 of 2021, Lyft served 17.1 million active riders through our suite of mobile applications. At this scale, every crash, frozen frame, or hiccup can translate to thousands of hours of wasted time and frustration. Given the potential to impact millions of Lyft’s customers, we doubled our efforts in 2020 to improve our applications’ speed and stability by launching an internal mobile performance initiative.
This is the first in a series of stories detailing our journey to materially improve the speed and stability of our applications. We hope that this can be used by other companies as a reference and inspiration for their own investments in this space.
In this post, we will explore both how and why we started investing in mobile performance at Lyft, and we will dig into our team’s philosophy — everything from how we select projects to how we evaluate success.
Defining Performance
Performance within the context of software development has historically referred solely to the speed of an application’s experiences (i.e., app start, and overall “smoothness” of the experience). At Lyft we see the investment in performance as an effort to remove unexpected deviations in functionality or latency.
Mobile Performance = Speed (app launch, smoothness of experience) + Stability (crash rate, bugs, etc.)
The Early Days
Before we kick-started our mobile performance initiative in Q1 of 2020, product teams primarily used an app crash metric to indicate performance. We capture app crashes on the client and ingest them into our analytics pipeline. If the crash rate exceeds a threshold, it triggers alarms, and feature developers can use attached crash reports to dig into the crash details via Bugsnag.
Outlined above is perhaps the most common process used at most mobile tech companies to evaluate performance, but is there more to it? Are there other performance metrics we should track outside of crashes? Is the crash rate we track truly accurate? How do our apps compare to the rest of the mobile technology industry? At the time, we did not have the tools or the technology necessary to answer these seemingly basic performance questions. As Lyft scaled to 150+ mobile engineers, it became increasingly important to care about the deeper details of performance.
Taking the First Step
Deep investments in performance can be difficult to prioritize. This is the case for a number of reasons:
- Correlating performance to core business metrics can be challenging.
- There are many other areas of investment which tend to take precedent, including product/feature changes, architecture, experimentation, observability, etc.
- Performance investments tend to require deep knowledge and lengthy investigations, which requires a large upfront cost with delayed results.
Many times, the hardest part of a journey is the first step, and we decided to prove out the value of continued investment in mobile performance by finding one problem, fixing it, and demonstrating the value of that fix.
Scoping the Project
Luckily, during the development of a project unrelated to performance, we had already identified one screen in particular that we theorized was causing a drastic increase in the Time-To-Interact (TTI) of the Lyft Driver app. We used this insight to propose a performance project that would optimize this screen with the overall goal of improving our application’s TTI.
While both the Android and iOS Lyft applications could benefit from the optimization we identified, we chose to focus on Android first. Then, if the results proved promising, we would pursue the same optimization in our iOS app.
The Proposal
We conducted some initial research to understand the importance of TTI in mobile applications. We compiled our research into the following categories:
- It’s a better user experience for our customers: TTI is so important that both Google and Apple have dedicated extensive documentation resources on how to improve app startup time and why it makes a difference.
- Positive correlation between low startup time and business metrics: There is a reason that so many companies focus on improving app startup time — it has an impact on business metrics. So much so that many companies have publicly outlined their approaches to improvements: Facebook, LinkedIn, Redfin, NYTimes.
- Startup time is a factor in how Google rank applications in app store searches: Android Vitals performance metrics are used for ranking within Google Play searches. If we want to be a leader on the app store, we have to be a leader in performance.
We used the Google Play Console to understand that in comparison to our existing peer group in the Play Console (10 applications), we are around +15–20% slower in Google’s definition of app startup time. We combined the project idea with this rationale to form a proposal, and presented it to leadership. The proposal was convincing enough to draw consensus, and we were given the green light to execute.
Build and Launch
When a mobile application launches, there are numerous phases that occur to prepare for interaction from the user. We started by profiling each phase of app launch in an effort to understand where the largest opportunities were. Below are the phases as well as some initial data gathered via local profiling:
- Application: Start the application process.
- Activity: Start the application UI rendering.
- Bootstrap: “Bootstrap” the app with network requests and data necessary to render the home screen.
- Display the home screen: Show the main screen for drivers to operate on.
The results verified our hypothesis that the bootstrap screen in particular represented the majority of app launch time. For context, the bootstrap screen is used on app launch primarily as a placeholder for engineering to “bootstrap” the app with network requests and data necessary to render the app. From a user perspective, drivers just wait on a screen that displays the Lyft logo.
We were able to identify a number of approaches to optimize performance of the bootstrap screen.
- Reduce network calls on the critical launch path: After decomposing our backend services in 2019, some network calls in the launch path were no longer required, so we removed them.
- Execute network calls asynchronously if possible: If some data was still required for our application to function but was not needed during app launch, these calls were made non-blocking to allow the launch to proceed without them. For example, the blocking network call which fetched whether the user enabled the Shortcut feature or not was safely moved to be non-blocking in the background.
- Cache data between sessions: Some data was necessary for app launch, but does not change over time. We began caching this data between sessions, further reducing overall TTI. For example, depending on whether a driver has started the driver application or not, the Lyft driver app shows different landing pages after app launch. Once a driver has started the application, the value would not change for that driver. So we can cache the data until the driver logs out.
These simple improvements were estimated to improve TTI for the vast majority of drivers. Using the information above, we expected to decrease TTI by >50% in many cases. After estimating the opportunity size and feasibility, we made the changes.
Before launching the optimizations to all drivers, we again wanted to quantify the improvements. We rolled out the change behind an A/B experiment and introduced a new TTI metric in our A/B testing framework so we could analyze the results.
Results
The Android experiment results showed a dramatic 21% (3.7s) average reduction in app startup TTI, and a 5% increase in driver sessions. The same experiment later on iOS showed an even more dramatic 42.2% (3.2s) average reduction. A before and after GIF can be seen above. We then shared these results with leadership along with a proposal for continued investment, which produced enough buy-in to create a dedicated mobile performance work stream.
Learnings
- Pick the most actionable project. Delivering significant results on a brand new work stream is critical to achieve consensus for further investment, so ensure the first project intersects at the apex of highest impact and least effort.
- Communicate early and often. Early stage initiatives are not as well known throughout the company as those that are more established. Consistent communication of goals, plans, experiments, and results is key to garnering support from leadership. We also found it beneficial to spend time educating product, leadership, and the rest of the organization on the problem space and how we are tackling it, since much of the work is highly technical.
Up Next
We chose to improve TTI because we knew it was the most actionable project that could produce meaningful results and offered the most compelling justification for further investment; but that was just the start. In our next blog post in this series, we will outline the next phase of our mobile performance initiative: tackling stability issues.
If you’re interested in working on performance at Lyft, we’re hiring!
