构建可投入生产的Agentic系统：来自Shopify Sidekick的经验教训
This blog post is based on the talk presented by Andrew McNamara, Ben Lafferty, and Michael Garner at ICML 2025: Building Production Ready Agentic Systems: Architecture, LLM-based Evaluation, and GRPO Training.
这篇博客文章基于 Andrew McNamara、Ben Lafferty 和 Michael Garner 在 ICML 2025 上的演讲：Building Production Ready Agentic Systems: Architecture, LLM-based Evaluation, and GRPO Training。
At Shopify, we've been building Sidekick, an AI-powered assistant that helps merchants manage their stores through natural language interactions. From analyzing customer segments to filling product forms and navigating complex admin interfaces, Sidekick has evolved from a simple tool-calling system into a sophisticated agentic platform. Along the way, we've learned valuable lessons about architecture design, evaluation methodologies, and training techniques that we want to share with the broader AI engineering community.
在 Shopify，我们一直在构建 Sidekick，这是一款通过自然语言交互帮助商家管理店铺的 AI 助手。从分析客户细分到填写商品表单、导航复杂的管理界面，Sidekick 已从简单的工具调用系统演进为成熟的智能体平台。在此过程中，我们在架构设计、评估方法论和训练技术方面积累了宝贵经验，希望与更广泛的 AI 工程社区分享。
The Evolution of Sidekick's Architecture
Sidekick 架构的演进
Sidekick is built around what Anthropic calls the "agentic loop" – a continuous cycle where a human provides input, an LLM processes that input and decides on actions, those actions are executed in the environment, feedback is collected, and the cycle continues until the task is complete.
Sidekick 围绕 Anthropic 所称的“智能体循环”构建——人类提供输入，LLM 处理并决定行动，行动在环境中执行，收集反馈，循环继续直至任务完成。
In practice, this means Sidekick can handle requests like "which of my customers are from Toronto?" by automatically querying customer data, applying the appropriate filters, and presenting results. Or when a merchant asks for help writing SEO descriptions, Sidekick can identify the relevant product, understand the context, and fill in optimized content directly into the product form.
在实践中，这意味着Sidekick可以处理诸如“我的哪些客户来自多伦多？”的请求，通过自动查询客户数据、应用适当的筛选条件并呈现结果。或者当商家请求帮助撰写SEO描述时，Sidekick可以识别相关产品、理解上下文，并直接将优化内容填入产品表单。
