牧羊人：Stripe如何将Chronon调整为适应机器学习特性的开发规模

Machine learning (ML) is a foundation underlying nearly every facet of Stripe’s global operations, optimizing everything from backend processing to user interfaces. Applications of ML at Stripe add hundreds of millions of dollars to the internet economy each year, benefiting millions of businesses and customers worldwide. Developing and deploying ML models is a complex multistage process, and one of the hardest steps is feature engineering.

机器学习（ML）是支撑Stripe全球运营几乎所有方面的基础，优化从后端处理到用户界面的一切。Stripe在ML的应用每年为互联网经济增加数亿美元，惠及全球数百万企业和客户。开发和部署ML模型是一个复杂的多阶段过程，其中最困难的步骤之一是特征工程。

Before a feature—an input to an ML model—can be deployed into production, it typically goes through multiple iterations of ideation, prototyping, and evaluation. This is particularly challenging at Stripe’s scale, where features have to be identified among hundreds of terabytes of raw data. As an engineer on the ML Features team, my goal is to build infrastructure and tooling to streamline ML feature development. The ideal platform needs to power ML feature development across huge datasets while meeting strict latency and freshness requirements. 

在将特征（ML模型的输入）部署到生产环境之前，通常需要经历多次构思、原型设计和评估迭代。在Stripe的规模下，这是一个特别具有挑战性的任务，因为需要从数百TB的原始数据中识别出特征。作为ML特征团队的工程师，我的目标是构建基础设施和工具，以简化ML特征的开发流程。理想的平台需要能够在大规模数据集上支持ML特征的开发，并满足严格的延迟和新鲜度要求。

In 2022 we began a partnership with Airbnb to adapt and implement its platform, Chronon, as the foundation for Shepherd—our next-generation ML feature engineering platform—with a view to open sourcing it. We’ve already used it to build a new production model for fraud detection with over 200 features, and so far the Shepherd-enabled model has outperformed our previous model, blocking tens of millions of dollars of additional fraud per year. While our work building Shepherd was specific to Stripe, we are generalizing the approach by contributing optimizations and new functionality to Chronon that anyone can use.

2022年，我们与Airbnb建立了合作伙伴关系，以其平台Chronon为基础，开发和实施了Shepherd——我们的下一代机器学习特征工程平台，并计划将其开源。我们已经使用它构建了一个新的欺诈检测生产模型，拥有超过200个特征，并且到目前为止，基于Sheph...