高质量LLM基础的Dasher支持自动化之路

The independent contractors who do deliveries through DoorDash - "Dashers" - pick up orders from merchants and deliver them to customers. That process, however, is far more complex than it appears to be on the surface. Dashers, particularly those new to the process, sometimes need support to resolve issues they encounter in the delivery process. This post describes how we have worked to improve the existing Dasher support system using large language models, or LLMs, and a retrieval augmented generation system, or RAG. We also describe how we manage the improved system with LLM Judge, LLM Guardrail, and quality evaluation.

通过DoorDash进行配送的独立承包商——“Dashers”——从商家处提取订单并将其送达客户。然而，这个过程远比表面上看起来复杂。Dashers，特别是那些新手，有时需要支持来解决他们在配送过程中遇到的问题。本文描述了我们如何利用大型语言模型（LLMs）和检索增强生成系统（RAG）来改善现有的Dasher支持系统。我们还描述了如何通过LLM Judge、LLM Guardrail和质量评估来管理改进后的系统。

When Dashers encounter difficulties during delivery, they can reach out to DoorDash support. We provide automated solutions and can connect Dashers to human support agents as needed. The automated support system typically resolves issues faster than human agents can because Dashers are connected instantly to a system that provides answers at digital speed. Our existing automated support system, however, provides flow-based resolutions, relying heavily on pre-built resolution paths. This means that only a small subset of Dasher issues can be resolved quickly. Although we offer Dashers a good collection of articles in our knowledge base, three issues limit their usefulness:

当Dashers在交付过程中遇到困难时，他们可以联系DoorDash支持。我们提供自动化解决方案，并根据需要将Dashers连接到人工支持代理。自动支持系统通常比人工代理更快地解决问题，因为Dashers可以立即连接到一个以数字速度提供答案的系统。然而，我们现有的自动支持系统提供基于流程的解决方案，严重依赖于预构建的解决路径。这意味着只有一小部分Dasher问题可以快速解决。尽管我们为Dashers提供了良好的知识库文章集合，但有三个问题限制了它们的有效性：

It can be difficult to find the relevant article;
找到相关的文章可能很困难；
It takes time to find useful information within any particular article;
在任何特定文章中找到有用信息需要时间；
The articles are all in English, but many Dashers prefer a different language.
所有文章都是英文的，但许多Dashers更喜欢其他语言。

These problems form a ...