知鸦日报2024-06-18

当你需要更精准的控制 LLM 的输出结果和定义其执行逻辑时，该怎么做？

随着前端业务的日趋庞大，及时发现和解决业务中的问题、优化用户体验、实时监控业务健康度变得愈发重要。

Pinterest在构建Ray基础架构时，采用了中间层解决方案，包括API Server、Ray Cluster / Job Controller和MySQL数据库。该解决方案简化了用户与Kubernetes的交互，并提供了实时的Ray集群和Ray Job状态监控。Pinterest还使用AWS S3持久化Ray日志，并在Ray Cluster UI上展示。他们还开发了Statsboard工具，用于展示Ray应用程序的性能指标和特定功能。此外，Pinterest还提供了三种开发Ray应用程序的选项，并通过ML RESTful API支持Dev server、Jupyter和Spinner工作流。Pinterest还提供了两种测试选项，即Unittest和Integration Test，以及网络和安全性方面的考虑。

本文介绍了在Linux平台上适配开发QQ音视频通话的过程。作者调研了Linux平台的特点和常见的发行版，选择了x64和arm64架构进行适配。还讨论了不同发行版的软件包管理系统和常见的软件包格式。以桌面版本QQ为例，打包了deb、rpm和AppImage的软件包格式。此外，在SDK开发中，根据不同平台提供了静态库和动态库。

We previously shared our insights on the tactics we have honed while operating LLM applications. Tactics are granular: they are the specific actions employed to achieve specific objectives. We also shared our perspective on operations: the higher-level processes in place to support tactical work to achieve objectives.

But where do those objectives come from? That is the domain of strategy. Strategy answers the “what” and “why” questions behind the “how” of tactics and operations.

A possibly apocryphal quote attributed to many leaders reads: “Amateurs talk strategy and tactics. Professionals talk operations.” Where the tactical perspective sees a thicket of sui generis problems, the operational perspective sees a pattern of organizational dysfunction to repair. Where the strategic perspective sees an opportunity, the operational perspective sees a challenge worth rising to.

In part 1 of this essay, we introduced the tactical nuts and bolts of working with LLMs. In the next part, we will zoom out to cover the long-term strategic considerations. In this part, we discuss the operational aspects of building LLM applications that sit between strategy and tactics and bring rubber to meet roads.

It’s an exciting time to build with large language models (LLMs). Over the past year, LLMs have become “good enough” for real-world applications. The pace of improvements in LLMs, coupled with a parade of demos on social media, will fuel an estimated $200B investment in AI by 2025. LLMs are also broadly accessible, allowing everyone, not just ML engineers and scientists, to build intelligence into their products. While the barrier to entry for building AI products has been lowered, creating those effective beyond a demo remains a deceptively difficult endeavor.

We’ve identified some crucial, yet often neglected, lessons and methodologies informed by machine learning that are essential for developing products based on LLMs. Awareness of these concepts can give you a competitive advantage against most others in the field without requiring ML expertise! Over the past year, the six of us have been building real-world applications on top of LLMs. We realized that there was a need to distill these lessons in one place for the benefit of the community.

基于日志中台实时架构实现准确、可扩展、低成本的日志埋点监控。

流量回放平台是将生产环境的流量录制下来在线下环境进行mock或不mock回放，但流量回放不能很好的细化接口规则，无法有效组织和验证业务接口内的预期表现，针对这一痛点，我们急需一个“构建智能规则验证流程，一键直达验证结果”的平台。

记录一次升级的流程。

天猫国际用户Push中心承接国际用户触达相关需求，如短信、消息投放等，存在较高并发场景。该系统曾发现一个查询投放计划为null的异常情况，初期排查毫无头绪，后来灵光乍现，原是缓存一致性问题！

科学道歉，实现和解。

滂臭的拖鞋，真的不能全怪脚。