使用 RAG 解锁视频的知识共享

At Vimeo, we’re highly invested in providing the most powerful way for our users to pull value from their videos. We believe that one effective way to achieve this is by enabling our viewers to converse with video content in natural language. While in the past, such a task could have been deemed as too ambitious, generative AI has revolutionized the field and has made such chat interactions almost commonplace.

在Vimeo，我们非常致力于为用户提供从视频中获取价值的最强大方式。我们相信，实现这一目标的一种有效方式是让我们的观众能够用自然语言与视频内容进行对话。虽然在过去，这样的任务可能被认为过于雄心勃勃，但生成式人工智能已经彻底改变了这个领域，并使这种聊天交互几乎成为常态。

Consider, for example, the 15-minute video of Steve Jobs’ 2005 Stanford Commencement Address. Say you haven’t watched the video before, and you don’t have 15 minutes to watch it now, but you need a summary of the information that the video contains. You can start by asking our video Q&A: “What is the video about?” In return you get a brief description of the video and some relevant playable moments that you can click to watch. You also get some related questions that you can select to dig deeper into any given topic, or you can type new questions of your own. See Figure 1 for details.

例如，考虑一下史蒂夫·乔布斯2005年斯坦福大学毕业典礼演讲的15分钟视频。假设您以前没有看过这个视频，现在也没有15分钟的时间来观看它，但您需要一个包含视频信息摘要。您可以通过询问我们的视频问答系统开始：“这个视频是关于什么的？”然后您会得到一个简短的视频描述和一些相关的可播放片段，您可以点击观看。您还会得到一些相关问题，您可以选择深入研究任何给定的主题，或者您可以输入自己的新问题。有关详细信息，请参见图1。

Figure 1. The Vimeo video Q&A system can summarize the content of a lengthy video instantly, link to key moments, and suggest additional questions.

图1。 Vimeo视频问答系统可以即时总结冗长视频的内容，链接到关键时刻，并提出额外的问题。

The Vimeo video Q&A system is implemented using one of the most recent developments in generative AI called retrieval augmented generation, or RAG for short. This method has become so popular lately that many, including NVIDIA, Custom GPT, and ZS, have crowned 2024 as “the year of RAG.” In this blog post, we present how we apply the RAG approach to videos and explore some of the unique problems we face in this specific domain.

Vimeo视频问答系统采用了最新的生成式AI发展之一，称为检索增强生成（RAG）。这种方法最近变得非常流行，包括NVIDIA、Custom GP...