在LLM领域的导航：Uber与GenAI Gateway的创新

Large Language Models (LLMs) have emerged as pivotal instruments in the tech industry, unlocking new avenues for innovation and progress across various sectors. At Uber, the impact of LLMs is particularly noticeable, with over 60 distinct use cases being identified in diverse domains, ranging from process automation to customer support and content generation. As teams at Uber embark on the journey of integrating LLMs into their products, several challenges have surfaced. Notably, the disparate integration strategies adopted by different teams have led to inefficiencies and redundant efforts.

大型语言模型（LLMs）已成为科技行业的关键工具，在各个领域中开启了创新和进步的新途径。在Uber，LLMs的影响尤为明显，已经确定了60多个不同的用例，涵盖了从流程自动化到客户支持和内容生成的各个领域。在Uber的团队将LLMs整合到他们的产品中的过程中，出现了一些挑战。值得注意的是，不同团队采用的不同集成策略导致了低效和冗余的工作。

To address these challenges and harness the growing demand for LLMs, Uber’s Michelangelo team has innovated a solution: the GenAI Gateway. The GenAI Gateway serves as a unified platform for all LLM use cases within Uber, offering seamless access to models from various vendors like OpenAI and Vertex AI, as well as Uber-hosted models, through a consistent and efficient interface. The GenAI Gateway is designed to simplify the integration process for teams looking to leverage LLMs in their projects. Its easy onboarding process reduces the effort required by teams, providing a clear and straightforward path to harness the power of LLMs. In addition, a standardized review process, managed by the Engineering Security team, reviews use cases against Uber’s data handling standard before use cases are granted access to the gateway.  If testing is successful these projects go through our standard, cross-functional software development process. The centralized nature of the gateway also streamlines the management of usage and budgeting across various teams, promoting greater control and operational efficiency in the integration of LLMs across the company.

为了应对这些挑战并满足对LLM的不断增长的需求，Uber的Michelangelo团队创新了一种解决方案：GenAI Gateway。GenAI Gateway作为Uber内部所有LLM用例的统一平台，通过一致高效的接口，为来自Op...