我们从一年的LLM构建中学到了什么（第二部分）

A possibly apocryphal quote attributed to many leaders reads: “Amateurs talk strategy and tactics. Professionals talk operations.” Where the tactical perspective sees a thicket of sui generis problems, the operational perspective sees a pattern of organizational dysfunction to repair. Where the strategic perspective sees an opportunity, the operational perspective sees a challenge worth rising to.

一句可能是虚构的引述被归功于许多领导者，它说：“业余人士谈论战略和战术。专业人士谈论运营。”战术视角看到了一片杂乱的独特问题，而运营视角则看到了组织功能障碍的模式需要修复。战略视角看到了机会，而运营视角则看到了值得应对的挑战。

Learn faster. Dig deeper. See farther.

学得更快。挖得更深。看得更远。

In part 1 of this essay, we introduced the tactical nuts and bolts of working with LLMs. In the next part, we will zoom out to cover the long-term strategic considerations. In this part, we discuss the operational aspects of building LLM applications that sit between strategy and tactics and bring rubber to meet roads.

在本文的第一部分中，我们介绍了与LLM合作的战术性细节。在下一部分中，我们将放大视野，讨论长期的战略性考虑。在本部分中，我们将讨论在战略和战术之间的LLM应用的操作方面，以实现战略目标。

Operating an LLM application raises some questions that are familiar from operating traditional software systems, often with a novel spin to keep things spicy. LLM applications also raise entirely new questions. We split these questions, and our answers, into four parts: data, models, product, and people.

运行 LLM 应用程序会引发一些熟悉的问题，这些问题在操作传统软件系统时常常出现，但也有一些新的问题。我们将这些问题和我们的答案分为四个部分：数据、模型、产品和人员。

For data, we answer: How and how often should you review LLM inputs and outputs? How do you measure and reduce test-prod skew? 

对于数据，我们回答了以下问题：如何以及多久应该审查 LLM 的输入和输出？如何衡量和减少测试与生产之间的偏差？

For models, we answer: How do you integrate language models into the rest of the stack? How should you think about versioning models and migrating between models and versions?

对于模型，我们回答以下问题：如何将语言模型整合到整个堆栈中？如何考虑模型和版本之间的版本控制和迁移？

For product, we answer: When should design be involved in the application development process, and why is it “as early as possible”? How do you design user experiences with rich human-in-the-loop feedback? How do you prioritize the many conflicting requirements? ...