Zalando 的 Agentic Engineering：概览

While the landscape and environment rapidly changes every day, the whole industry is figuring out how to approach Agentic Engineering. With more than 250 engineering teams innovating across our business lines, we're observing value and impact of LLMs in different forms and at different paces. We shared some early wins on using LLMs for product data enrichments, LLM as a judge for improving search quality, relevance assessment in product search, or frontend migrations.

随着格局和环境每天都在快速变化，整个行业都在探索如何推进 Agentic Engineering。我们拥有超过 250 个工程团队在各个业务线进行创新，我们正以不同的形式和节奏观察 LLMs 的价值和影响。我们分享了一些早期成果，包括使用 LLMs 进行商品数据丰富，将 LLM 作为评判者来提升搜索质量、商品搜索中的相关性评估，或进行前端迁移。

Recently, we have been looking back at our progress in the past 2.5 years and we'd like to share a few approaches that worked well for us.

最近，我们回顾了过去 2.5 年的进展，并希望分享一些对我们行之有效的方法。

LLM proxy for API-based LLM access from day 1

LLM 代理：从第一天起支持基于 API 的 LLM 访问

We've been a GitHub Copilot user from the early days when it offered autocomplete in the IDE. To complement this offering and provide API-based access to LLMs, our ML platform team deployed in January 2024 a LiteLLM based API proxy with access to models from different providers (now: OpenAI, AWS Bedrock, and Google Vertex). This way, it became easy for our engineers to experiment with different tools and models. The platform team got a single point to measure adoption via: MAU, WAU, model, User-Agent.

从 GitHub Copilot 在 IDE 中提供自动补全功能的早期开始，我们就一直是它的用户。为了补充这一产品并提供基于 API 的 LLMs 访问，我们的 ML 平台团队于 2024 年 1 月部署了一个基于 LiteLLM 的 API 代理，可访问来自不同提供商的模型（目前包括：OpenAI、AWS Bedrock 和 Google Vertex）。这样一来，我们的工程师就可以轻松地尝试不同的工具和模型。平台团队获得了一个单一指标点来衡量采用情况：MAU、WAU、模型、User-Agent。

We like LiteLLM for its extensibility. We use post-call hooks for anonymized cost tracking and pre-call hooks for enforcing client version upgrades by restricting access to the proxy based on the User-Agent header. For self-managed client installations, unfortunately blocking access is the only effective measure. Same goes for retiring models. There is always a long-tail group of users...