The Recommendation System at Lyft
Recommendation plays an important role in Lyft’s understanding of its riders and allows for customizing app experiences to better fulfill their needs. At times, recommendations are also leveraged to manage the marketplace, making sure there’s a healthy balance between ride demand and driver supply. This allows ride requests to be fulfilled with more desirable dispatch outcomes such as matching riders with the best driver nearby.
This blog post focuses on the scope and the goals of the recommendation system, and explores some of the most recent changes the Rider team has made to better serve Lyft’s riders.
Introduction: Scope of the Recommendation System
The recommendation system covers user experiences throughout the ride journey.
Screenshots are illustrative. May not capture the current experience.
One-Tap module on the home screen: At app open, riders can skip the entire request flow and request a ride with a tap at a button. This is available for high frequency users that have extensive travel history with Lyft. This is a convenient addition to the home screen and saves time and energy for users who are in a rush.
Ranking & Preselection on the mode selector: After the user sets their destination, the app presents a ranked list of product offerings based on the user’s travel preferences and the current marketplace conditions. Some visual highlights are also displayed to help clarify the tradeoffs across different options. For instance, Lyft highlights the “fastest” and “your usual” ride types to enable riders to make faster and easier decisions when evaluating their options.
Post request cross-sells: Marketplace conditions are dynamic, and mode availability, prices and ETAs (i.e. estimated time of arrival) can change very quickly. In select sessions, these changes can be captured and made beneficial to riders in our post request cross-sell experience, where an interstitial prompt is introduced detailing upgrade options with a better ETA or price.
This blog mostly focuses on the mode selector to explain how rankings have evolved in the past years and briefly touches on the post request cross-sells.
What Does the Recommendation System Do?
Years ago, when mode options were limited, the mode list was organized simply through ranking by price or ETA. Additionally, in certain regions, the mode list was static in order to observe rider behaviors as a baseline.
For the past two years, Lyft has invested in improving customization of mode recommendations. Since then, the recommendation system has proven to drive more desirable outcomes compared to a static system, by solving problems in three major areas:
- The challenge of overchoice
- The cold start problem for new modes
- Dynamic business and user experience goals
The challenge of overchoice
As shown in the mock below, these are a subset of offerings provided to users across Lyft available regions. It’s uncommon to see all of them appearing at the same time, but in larger markets like San Francisco or New York, users can easily find themselves in front of at least 8–10 options across rideshare, bikes, scooters and rentals. As Lyft’s portfolio grows, a typical rider can have a hard time discovering and understanding the wide variety of products that Lyft has to offer, which may result in riders accidentally booking the wrong mode.
Screenshots are illustrative. May not capture the current experience.
To resolve this, the recommendation system leverages a set of machine learning models to predict a rider’s propensity of converting into each mode and customizes the rankings based on it. Rich information has been considered in building these models, including temporal features like location and time info, supply / demand signals, ride histories and user preferences.
- Algorithm: LightGBM (each mode is considered as a distinct class, with weights determined by analyzing mode specific financial metrics)
- Model objective: lambda rank or multi-class classification, dependent on different use cases
- Hyperparameters: a lot of tuning on the typical hyperparameters like maximum depth and learning rate is required to achieve desirable results. Specifically, Lyft’s in-house distributed hyperparameter optimization pipeline is used for the majority of its business critical models.
Screenshots are illustrative. May not capture the current experience.
The modes above the fold and the preselection are powered by the set of model results. When users initially land on the mode selector, the number of modes shown is limited to 3–4 above the fold (see mock above). This is to reduce cognitive overload for users, and have them focus on modes that would best represent their transportation needs. Of course, users would still have the ability to swipe up and explore other options below the fold, which are ranked based on categories and pricing.
In addition to ranking, preselection helps reduce steps in our ride request flow. In 2019, the user’s last mode taken was preselected. This captured user stickiness for some use cases, but introduced natural exploration bias. That said, in 2020, Lyft moved towards a more user centric approach — preselecting a user’s most frequently used mode. This was more robust to outliers but could be jumpy, especially for infrequent users. In 2022, Lyft aimed to solve this by moving towards a model-based approach, preselecting the mode with the highest predicted propensity score. This solution is by design real-time, dynamic and has proven to be more accurate and effective.
The cold start problem for new modes
Ranking of the options highly impacts a rider’s mode choices. Therefore, new offerings with limited user touch points should still be recommended properly to gain awareness and habituation. When Wait & Save and Shared products were first introduced, they got very limited visibility above the fold since no or limited data was at our disposal to inform rankings. In the recommendation system today, a post-processor was introduced as an additional layer to adjust the machine learning model results, mitigating the natural bias created by the lack of sufficient training data. This way, Lyft’s recommendation system can sufficiently and effectively make the strategic decisions to promote certain options without making substantial system changes.
Dynamic business and user experience goals
Rides and conversions — reflected in the increasing accuracy rates in model predictions — are not always the best goals to optimize for in the recommendation system. To create a seamless request experience and ensure a more balanced marketplace, other metrics like ride reliability and supply / demand balance need to be actively monitored. This can be achieved in a myriad of ways. One example is the selective cross-sell experience following a ride request. The triggering of this experience is not very common, but can be an effective lever for elevating the matching efficiency and smoothing demand. However, this doesn’t mean that the user experience becomes secondary — rider propensity, price and wait time trade-offs remain critical guardrails for determining eligibility for these mode cross-sells.
In summary, the recommendation system uses a hybrid approach towards personalizing rankings and mode suggestions. Machine learning propensity models serve about 90% of the use cases and other layers of adjustments are applied to optimize for additional business and user goals.
Upcoming Improvements
In late 2022, Lyft introduced the “one-tap” modules, a seamless experience that highlights the best options (both destinations and modes) for riders upon opening the app, speeding up the request experience. The coverage of “one-tap” will be expanded to include more relevant use cases and make it easily accessible to a wide range of users.
Introduce reinforcement learning
Lyft’s in-house contextual bandit system will enable more dynamic user-system interactions to be considered and react in a more real-time fashion (go to this blog to learn more about its first application in pricing). Longer term user engagement behaviors can be taken into account too to resolve any long term effects uncaptured by the existing manual training of models.
Are You as Excited as We Are?
Interested in applying science at scale and learning more about how data-driven decisions are made at Lyft? Check out more exciting blogs in Lyft Data Science. If you find this blog helpful and want to discuss more, please don’t hesitate to reach out via email.