Introducing Carbon Feed for Earners: The One-Stop Info Shop

After launching the Driver App in 2018 to over 2 million earners worldwide, we added content and functionality at a rapid pace. Although this really bolstered the platform, allowing for high-density and high-frequency content, and provided drivers and couriers with new ways to earn, it came with significant costs. For earners, it became increasingly difficult to find the information that they needed to make key decisions about work at any given time. For internal teams within Uber trying to emit information, there was pressure to be loud across multiple channels in order to reach earners effectively; teams had to, in essence, compete against one another to ensure that their intended messages reached the right audiences. As a result, collective goals were inadvertently superseded by individual teams lacking context.

We created Carbon Feed to serve these 2 primary stakeholders: For earners, Feed should be the one-stop shop for all the information they need to make effective decisions. Having a single, unified surface where earners can compare their different options at a given time and assess the tradeoffs would be inherently valuable. For internal teams, Feed should enable them to easily convey useful information to earners through a visible, central surface. Making the process seamless by requiring teams to only go through a single, streamlined integration would help them communicate more effectively.

From its early days, the Feed was a wild success in fulfilling its 2 aforementioned goals. Its impact became evident through a global holdout experiment. In the holdout, some earners were shown the Feed, while others were still shown the Daily Brief (one of Feed’s predecessors). From the experiment, it was concluded that the Feed had a positive impact on KPIs, both for engagement and business metrics, compelling the Carbon Feed team to proceed with the global launch.

Since launch, Feed has continued its success. It has not only attracted teams from all across Uber to integrate with it—reaching a total of 70+ partner integrations—but it has also driven meaningful changes in both Supply and Peak Supply Hours. What is more, earners engage with the Feed meaningfully and regularly, with 88% of drivers that log into Carbon swiping up on Feed. Over 2.7 million unique drivers see Feed cards daily, generating a total of over 30 million daily unique Feed card impressions. 

Intelligent and Personalized

We believe that Feed should go beyond simply conveying information. It should be personalized to each earner, so that the most relevant information at that point in time gets the most prominent rank in each Feed. Without ranking, earners could waste time scrolling through the Feed to find the content they find most relevant. The Feed’s original random ranking was followed by ranking by a heuristic—the estimated financial impact each card would have for earners. This simple ranking increased driver engagement, measured as Click-Through Rate (CTR) compared to random, demonstrating clear benefits to ranking Feed.

After simply ranking positively affected engagement, we decided to further optimize CTR by showing earners’ Feeds that have been personalized using machine learning (ML). Specifically, we train a ML model to predict the likelihood of a click given an earner, a Feed card, and session context. With a likelihood score available for each card, we can construct the Feed for the target user in the current context. Since an earner’s relevance criteria changes depending on temporal and geospatial context, as well as previous Feed interactions, we use a variety of context, content, and prior activity-based features as input into our ML model. Using a ML model allows us to:

1. Prioritize content that matters to a given earner specifically, furthering our goal of providing earners with highly relevant information
2. Show content to relevant earner audiences for each partner team use case, furthering our goal of enabling internal teams to convey information efficiently
3. Provide a starting point for earner personalization, setting a new precedent and enhancing core algorithms’ sophistication

However, an accurate model is of little practical use if it cannot be updated over time, in response to an expanding universe of Feed content and changing earner needs. In order to build an adaptive ranking system, we created an offline ML lifecycle by leveraging Uber’s workflow orchestration tool, MLStudio. This end-to-end ML ecosystem involves feature engineering, experimentation, and analysis; model training, deployment, and serving; online/offline evaluation; and monitoring systems. With it, we’re able to update our features, as well as retrain, evaluate and deploy a freshly trained model on a daily basis, automatically.

We are constantly brainstorming, evaluating, and implementing changes to our ranking system—from input features to our optimization function. This quick iteration cycle is facilitated by utilizing Uber’s Michelangelo ML tools in concert with the Driver platform team’s content aggregation backend, called Carbon Aggregator.

Carbon Aggregator (written in Golang) has the ability to fetch content from various source-of-truth backend services in a hybrid fanout fashion. It started out by powering the Feed screen on the Uber Driver App, but since then has expanded to more screens. The aggregator talks to other backend services via a content API written in thrift maintained by our backend team. This allows the content creators to define the copy of the information card as well as interactions, scheduling and certain visual aspects. The information coming from these services range from the latest earner incentives, promotions, and insights, to in-app inbox messages, documents and compliance-related reminders, and other earnings opportunities. When an earner opens the Feed section in the Uber Driver app (Android/iOS), a request is made to the backend aggregator. The aggregator fetches certain candidates for Feed in real time in response to the client request, and certain candidates are stored beforehand, ready to be served. Once a final list of Feed candidates is prepared, the aggregator calls the online model, powered by the ranking modules described above, to get a click prediction score for each candidate. Cards are ranked based on their prediction scores before being sent to the presentation layer. In the presentation layer, the cards are converted into server-driven UI components that the mobile client expects. Once the mobile client receives the Feed payload it renders the cards according to a Server Driver component UI specification owned by the mobile platform team, ready for the earner to consume.

Near-Real-Time Features

In prior iterations, the Feed ranking model received features capturing the current session context from the feed aggregator request, but was limited in its capacity to process other intra-day information, as features are only updated daily. For example, a desirable behavior may be to decrease the click likelihood for a card that the earner has already clicked today. Since an earner’s context is constantly changing as they interact with the Carbon app throughout the day, it is critical that the model uses features which are updated frequently.

In order to have our ranking incorporate this type of intra-day information, we make use of near-real-time features. Uber’s Michelangelo Feature Store supports serving and management of such features after they are ingested from Kafka topics. We use Uber’s self-service streaming platform, AthenaX, to author streaming jobs, which consume application events and compute near-real-time features such as aggregated impressions and clicks. These features are then ingested by the feature store and made available immediately for online serving and daily for offline training.

Adding these features to our model improved earner engagement as measured by taps and impressions, while also making feed more dynamic, as evidenced by more content diversity in the top feed positions.

Looking Ahead

This is only the beginning for Feed. We are committed to continuously enhance the Feed’s capabilities, so as to further its promise to both earners and partner teams of being the one-stop shop for all critical information. We are actively working on streamlining the integration process for partner teams, making Feed more visually engaging and intuitively insightful through multiple card templates, and allowing earners to actively shape their Feeds. Our engineering design can already provide ranked feeds with low latency, and is scalable to dozens of cards and millions of earners, but we’re also focused on improving our ranking system.

As more Uber partner teams integrate with Feed to communicate with earners, the potential for the surface to impact other metrics across the platform increases. We aim to leverage multi-objective optimization to train a robust ranking model that can more holistically score different types of content. We also aim to optimize on longer term objectives, so that our ranking does not overfit on short term rewards. By constantly improving our ranking algorithms, we can give earners the most personally important information right when they want it.