Envoy Mobile is an ambitious open source initiative to bring the power of Envoy Proxy to mobile apps, leading to unparalleled observability, cutting edge technologies, control, and consistency in the mobile networking space.

Our journey of enabling user privacy rights started all the way back with the California Consumer Privacy Act (“CCPA”) going live in 2020. Recently, new state laws have begun going into effect, including upcoming changes in California with the California Privacy Rights Act, and our early work set us up for success. Lyft is positioned to seamlessly handle changes thanks to the choices we made with the federated design of our architecture. In this blog post, we’ll share an overview of some technical strategies Lyft uses to provide important privacy rights like those in CCPA and describe how we implemented user data export and deletion in our online systems.

To start off, let’s review some of the key challenges we faced in developing our response to user requests regarding privacy rights.

In our last blog, we discussed how managing our business effectively comes down to, in large part, making causally valid forecasts based on our decisions. Such forecasts accurately predict the future while still agreeing with experiment (e.g. increasing prices by X will decrease conversion by Y). With this, we can optimize our decisions to yield a desirable future.

But there remains a gap between theory and the implementation that makes it a reality. In this blog, we will discuss the design of software and algorithms we use to bridge this gap.

In a microservice world, significant route configuration changes to the front proxy are often required to keep up with an evolving business. Making these route configuration changes easily manageable and testable quickly becomes a challenging issue with the number of developers needing to make changes and the frequency of these changes.

Pricing forms the backbone of the Lyft’s marketplace system, which is core in the journey of improving people’s lives with the world’s best transportation. In general, the goal of price optimization is to find the price that balances supply and demand while covering the expenses necessary to provide an easy-to-use rideshare platform, where drivers have tremendous influence on the rider prices through their entry and exit on the platform.

Continuous Deployment (CD) is when software changes, such as new features and fixes, are automatically deployed to our customers as quickly and safely as possible. At Lyft, we pride ourselves on “making it happen”, so in 2019 we set out to move from our manual deployment process to CD. The microservice architecture at Lyft made our path to CD challenging, as adoption required changing how each team maintained and deployed their services. Not only did we have to make configuration changes to each service, but most importantly we had to drive cultural changes in how each team approached and performed their deployments. We recently hit a new milestone of 90+% of our approximately 1,000 services using CD to deploy to production. We would like to share the journey, technical details, and challenges we faced.

Efficiently managing our marketplace is a core objective of Lyft Data Science. That means providing meaningful financial incentives to drivers in order to supply affordable rides while keeping ETAs low under changing market conditions — no easy task!

Lyft’s tool chest contains a variety of market management products: rider coupons, driver bonuses, and pricing, to name a few. Using these efficiently requires a strong understanding of their downstream consequences — everything from counts of riders opening the Lyft app (“sessions”) to financial metrics.

To complicate the science further, our data is heavily confounded by our previous decisions, so a merely correlational model would fail us. Sifting out causal relationships is the only option for making smart forward looking decisions.

Machine Learning models at Lyft make millions of high stakes decisions per day from physical safety classification to fraud detection to real-time price optimization. Since these ML model based actions impact the real world experiences of our riders and drivers as well as Lyft’s top and bottom line, it is critical to prevent models from degrading in performance and alert on malfunctions.

However, identifying and preventing model problems is hard. Unlike problems in deterministic systems whose errors are easier to spot, models’ performance tends to gradually decrease, which is more difficult to detect.

Safety is fundamental to Lyft. Thus, as part of the Safety & Customer Care Web team, we strive to provide the safest experience for our users. Our focus is to bring safety and support features to our riders’ web experience (ride.lyft.com); this allows riders to request a ride from their browser without needing the mobile app. In doing so, we also maintain consistency with the mobile app experience.

Android applications such as Lyft’s apps are developed by a large number of contributors. This means that the codebase grows and changes very quickly. Features are constantly being added or improved, and all these modifications can potentially impact the performance of the application. Thus, it is important to understand how the application consumes CPU resources and to see the dynamics of such metrics across product releases.

We believe that relevant messaging is at the heart of our mission to improve people’s lives with the world’s best transportation.

In a previous blog post, we discussed LyftLearn’s infrastructure built on top of Kubernetes. In this post, we will focus on the compute layer of LyftLearn, and will discuss how LyftLearn solves some of the major pain points faced by Lyft’s machine learning practitioners.

We will focus on comparing Airflow and Flyte implementations at Lyft: dive into the architecture and summarize its benefits and drawbacks.

Lyft moves people through space and time. But where those people move, and why, is up to them. Lyft’s riders use our services to get to and from work, go out to dinner, visit family, and get to the airport. When and where they do so tells us a lot about urban mobility — whether and how the notions of neighborhood, geography, and landscape shape how people move through space.

Here we share what we can learn from long-run patterns on Lyft’s operations in major US cities. We see that cities vary a lot internally in how people travel, where, and when. That diversity implies a need for a diverse range of products and services. But, strikingly, we also see how cities resemble each other — that sometimes, common patterns, like urban downtowns, look more like other cities’ downtowns than they do their companion suburbs.

This is the fourth and final post in the series on how we scaled our development practices at Lyft in the face of an ever-increasing number of developers and services.

Vulnerability management is important, but can be incredibly time consuming. We have to scan our systems and then fix the vulnerabilities that we’ve discovered. In a large software engineering organization this becomes more challenging — service owners are responsible for fixing vulnerabilities in their systems along with all their other work, and security has to track this work, nudge engineers to actually fix things, and report to CISO/compliance/etc. Fortunately much of this work lends itself to automation, letting security engineers focus on understanding and fixing vulnerabilities! In this post we’ll focus specifically on web vulnerabilities, and some of the fun automation challenges this process poses.