Airbnb的AI驱动照片导览使用Vision Transformer

By: Pei Xiong, Aaron Yin, Jian Zhang, Lifan Yang, Lu Zhang, Dean Chen

作者： Pei Xiong, Aaron Yin, Jian Zhang, Lifan Yang, Lu Zhang, Dean Chen

Introduction

介绍

In recent years, the integration of artificial intelligence with travel platforms has transformed how people search for and book accommodations. As a leading global marketplace for unique travel experiences and accommodations, Airbnb constantly strives to enhance the guest experience by providing informative content about the variety of homes shared by our hosts. One of the ways we help guests better understand what a listing offers before they book is through our AI-powered photo tour feature.

近年来，人工智能与旅游平台的结合改变了人们搜索和预订住宿的方式。作为全球领先的独特旅行体验和住宿市场，Airbnb不断努力通过提供有关我们房东共享的各种房源的信息内容来提升客人的体验。我们帮助客人在预订前更好地了解房源提供的其中一种方式是通过我们的AI驱动的照片导览功能。

The AI-powered photo tour in the Listings tab, which helps hosts better organize their listing photos, leverages vision transformers’ fine-tuned feature to assess a diverse set of listing images and accurately identify and classify photos based into specific rooms and spaces. In this blog post, we will dive into the inner workings of the photo tour including model selection, pretraining, fine-tuning techniques, and the trade-offs between computational costs and scalability. We will also specifically discuss how we enhanced model accuracy despite having limited training data.

房源标签中的AI驱动照片导览，帮助房东更好地组织他们的房源照片，利用Vision Transformers的微调特性来评估多样化的房源图像，并准确识别和分类照片到特定的房间和空间。在这篇博客文章中，我们将深入探讨照片导览的内部工作原理，包括模型选择、预训练、微调技术以及计算成本和可扩展性之间的权衡。我们还将特别讨论在训练数据有限的情况下如何提高模型的准确性。

Figure 1: Photo Tour product powered by ML

图1：由机器学习驱动的照片导览产品

Methodology

方法论

Room Classification

房间分类

Room-type classification is the first aspect of the photo tour, The goal of room classification is to accurately categorize images into 16 different room types designed in the Airbnb product such as ‘Bedroom’, ‘Full bathroom’, ‘Half bathroom’, ‘Living room’, and ‘Kitchen’, providing users with a comprehensive understanding of the available spaces. The challenge lies in the diversity of roo...