How Airbnb Measures Listing Lifetime Value

A deep dive on the framework that lets us identify the most valuable listings for our guests.

By: Carlos Sanchez-Martinez, Sean O’Donnell, Lo-Hua Yuan, Yunshan Zhu

At Airbnb, we always strive to provide our community with the best experience. To do so, it’s important to understand what kinds of accommodation listings are valuable to our guests. We achieve this by calculating and using estimates of listing lifetime value. These estimates not only allow us to identify which types of listings resonate best with guests, but also help us develop resources and recommendations for hosts to increase the value driven by their listings.

Most of the existing literature on lifetime value focuses on traditional sales channels in which a single seller transacts with many buyers (e.g. a retailer selling clothing to a customer). In contrast, this blog post explains how we model lifetime value in a platform like Airbnb, with multiple sellers and buyers. In the first section, we describe our general listings lifetime value framework. In the second section, we discuss relevant challenges when putting this framework into practice.

Our Listing Lifetime Value Framework

Our listing lifetime value (LTV) framework estimates three different quantities of interest: baseline LTV, incremental LTV, and marketing-induced incremental LTV.

(1) Baseline LTV

To measure LTV, we need to define what we mean by “value” and what time horizon constitutes a “lifetime.” Simplifying slightly for the purposes of this blog post, we define and estimate our baseline listing LTV as the total number of bookings that a listing will make on Airbnb over the next 365 days.

We rely on machine learning and the rich information we have about our listings to estimate this quantity for each individual listing. In practice, we also follow financial guidance to arrive at present value by projecting outcomes into the future and applying a relevant discount rate to future value.

Table 1 shows some hypothetical baseline LTV estimates. As you can see from the examples, LTV is not static, and can evolve as we improve the accuracy of our estimates, observe changes in our marketplace, or even develop a listing (e.g., by providing guidance that helps hosts improve the listing to get more bookings).

Table 1. Example Listing LTV Estimates

We use baseline LTV estimates to segment our listings and identify which types of listings resonate best with our guests. This informs our supply expansion strategy. We also use baseline LTV to identify listings that are not expected to reach their full booking potential and may benefit from additional guidance.

(2) Incremental LTV

When estimating lifetime value, we face a challenge that is common across multi-sided marketplaces: the transactions made by one listing might come at the expense of another listing’s transactions. For example, when a new listing joins our marketplace, this listing will get some bookings from guests who were previously booking other listings. We need to account for this dynamic if we want to accurately measure how much value is added by each listing.

We address this challenge by creating “incremental LTV” estimates. We refer to the additional transactions that would not have occurred without the listing’s participation as “incremental value,” and the transactions that would have occurred even without the listing’s participation as “cannibalized value.” We estimate the incremental LTV for a listing by subtracting cannibalized value estimates from the baseline LTV. We explain this adjustment in more detail when discussing measurement challenges.

Figure 1. Cannibalization. In this context, cannibalization refers to the transactions that would have occurred even without a listing’s participation in the marketplace. For example, when a new listing joins the platform, some bookings obtained by that listing would have been made at other listings on the platform had the new listing not joined.

(3) Marketing-induced incremental LTV

Lifetime value is not static, and our LTV model needs to tell us how our internal initiatives bring additional listing value. For example, suppose we run a marketing campaign that provides hosts with tips on how to successfully improve their listings. To understand the return from the campaign, we need to measure how much value is accrued due to the campaign, and how much value would have been organically accrued without our marketing intervention. We calculate “marketing-induced incremental LTV” to measure how much additional listing LTV is created by our internal initiatives.

Having outlined our measurement framework (summarized in Figure 2), we now cover some of the technical challenges we faced when putting this framework into practice.

Figure 2. Listing LTV Framework

Challenges when measuring Listing Lifetime Value

The most important requirement for our framework is accurate estimation of baseline LTV. Figure 3 illustrates our estimation setup. First, we leverage listing features snapshotted at estimation time t. This data includes rich knowledge we have about each listing and host (availability, price, location, host tenure, etc). We then use these features to train our machine learning model. As a value label, we use the number of bookings made within the next 365-day period, which is observed on date t + 365.

Figure 3. Label vs. Feature Collection. Our label lands 365 days after we collect the initial set of features for our model.

This setup has two important implications that impact accuracy and evaluation:

• We have to wait 365 days to fully evaluate the accuracy of a prediction.
• Our initial training data might not allow us to make accurate predictions if we observe shocks between the time when the training data was captured, and the time when we score the model.

In practice, we felt the full consequences of these implications during the COVID-19 pandemic, when travel came to a halt and marketplace dynamics changed drastically. Our model’s training data from before the pandemic had dramatically different characteristics relative to the scoring data we collected after the pandemic. When dealing with this shock, we implemented various strategies that helped us improve model accuracy:

• Reducing training windows, allowing us to reduce model drift.
• Feeding the model with granular geographic data and human-provided information about external factors as borders closed and reopened due to the pandemic.
• Adopting LightGBM, which handles high cardinality features like the geographic variables mentioned previously.

Challenge (2): Measuring incrementality

Accounting for incrementality is challenging because we never observe the ground truth. While we observe how many bookings are made per listing, we cannot tell which bookings are incremental and which bookings are cannibalized from other listings.

Since we don’t have an incrementality label to estimate this outcome directly, we instead estimate a production function. Intuitively, incrementality is heavily dependent on our ability to connect both sides of our marketplace. Production functions allow us to identify when our supply of listings and demand from guests connect and provide incremental value. Incrementality estimates will be high when a segment has high guest demand and relatively low listing supply. In contrast, incrementality will be low when segments have a large volume of listing supply and relatively low demand, meaning guests have an easy time finding a place to stay and a new listing is more likely to cannibalize bookings from other listings.

Specifically, we model how our total supply of listings (S) and total demand from guests (D) impacts our target outcome bookings (O), as in equation (1):

We estimate this model with historical supply, demand, and outcome data aggregated across internally-defined segments that have little overlapping demand. Having estimated model (1), we calculate how extra supply of listings results in additional bookings in the given segment: this is our estimate of incrementality.

Challenge (3): Handling uncertainty

To handle the uncertainty we experienced during the pandemic, we began updating our LTV estimates as listings received greater or fewer numbers of bookings than initially expected. This approach has helped us capture any shocks that occur after making our initial predictions.

To show how this can be useful, let’s go back to our marketing campaign example. Assume that we run this campaign for six months, and that we measure the success of this campaign by comparing marketing-induced incremental LTV against our total marketing investment in the campaign. As a first approach, we could use the initial baseline LTV figures (which feed into marketing-induced LTV) estimated at the time when the listing was first targeted by our initiative. However, listings targeted on day 1 of the marketing campaign will have six months of booking history by the time the campaign ends and we evaluate success. A more accurate approach uses realized bookings after the initial prediction to start correcting for model error.

Table 2 illustrates how this works. Suppose that on 2024–01–01, we expect that Listing A will get a total of 16 bookings by the end of the year. If six months into the 365 day period, Listing A has received 16 bookings, we should adjust its expected value upward to, say, 21 bookings. In fact, every day for 365 days after 2024–01–01, we can look at the bookings that Listing A has accrued and adjust the expected bookings accordingly. By construction, the expected and accrued bookings converge to the final bookings 365 days after the initial booking date. Going back to our marketing example, if Listing A ultimately receives 20 bookings, updating the initial estimate means we went from 20% underprediction on day 0 to a more reasonable 5% overprediction as of month 6.

Table 2. Example of how we update listing lifetime value estimates.

In practice, we make daily adjustments to a listing’s expected value based on the listing’s accrued value, updated listing features, and value arrival patterns for similar listings estimated using historical data.

Conclusion

In this blog post, we explained how we approach listing lifetime value at Airbnb. We covered our measurement framework, including baseline LTV, incremental LTV, and marketing-induced incremental LTV. We also zoomed into measurement challenges, like when travel patterns changed drastically during the COVID pandemic and accurately estimating LTV became more difficult.

Estimating the lifetime value for each listing is important because it helps us serve our community more effectively. Use cases include:

• Identifying unique listing segments through which new hosts can showcase their hospitality to a large guest audience.
• Pinpointing locations where listings have an opportunity to get more bookings, and might benefit from additional demand.
• Identifying which internal marketing initiatives bring the most value to our community.

It’s also worth noting that our measurement framework may extend to other applications, such as the lifetime value for Airbnb Experiences listings, where the value of an experience listing will heavily depend on travel trends and on guests’ ability to discover these experiences.

We continue to solve interesting problems around LTV every day (and as more insights come up, we’ll keep sharing them on our blog). Can you see yourself making an impact here? If so, we encourage you to explore the open roles on our team.