重新设计Etsy的机器学习平台

Etsy leverages machine learning (ML) to create personalized experiences for our millions of buyers around the world with state-of-the-art search, ads, and recommendations. The ML Platform team at Etsy supports our machine learning experiments by developing and maintaining the technical infrastructure that Etsy’s ML practitioners rely on to prototype, train, and deploy ML models at scale. When we built Etsy’s first ML platform back in 2017, our data science team was growing, and a lack of mature, enterprise-scale solutions for managing the models they were building threatened to become a bottleneck for ML model delivery. We decided the best way forward was to build systems in-house that would present Etsy’s ML practitioners with a consistent, end-to-end interface for training, versioning, packaging, and deploying the models they were building in their preferred ML modeling libraries.

Etsy利用机器学习（ML），通过最先进的搜索、广告和推荐，为我们世界各地的数百万买家创造个性化体验。Etsy的ML平台团队通过开发和维护技术基础设施来支持我们的机器学习实验，Etsy的ML从业人员依靠这些技术基础设施来实现ML模型的原型化、训练和规模化部署。当我们在2017年建立Etsy的第一个ML平台时，我们的数据科学团队正在成长，而缺乏成熟的、企业级的解决方案来管理他们正在建立的模型，这有可能成为ML模型交付的瓶颈。我们决定最好的办法是在内部建立系统，为Etsy的ML从业者提供一个一致的、端到端的接口，用于训练、版本管理、打包和部署他们在首选ML建模库中构建的模型。

Figure 1: Different product teams interacting with Etsy’s ML Platform.

图1：不同的产品团队与Etsy的ML平台进行互动。

Our in-house solutions were written in 2017 when Etsy had a small data science team and we largely relied on a single implementation of logistic regression for production models. While our ML systems scaled well and were maintainable given Etsy’s size at the time, the maintenance cost started to grow with the increasing number and complexity of machine learning projects we had to support. In the industry, the number of third-party ML frameworks was exploding. It was becoming more difficult and costly to onboard new members of our data teams, who would have to drop tools they had already become proficient in to adapt themselves to our own V1 technology.  

我们的内部解决方案是在2017年编写的，当时Etsy有一个小的数据科学团队，我们主要依靠逻辑回归的单一实现来生产模型。虽然我们的ML系统扩展得很好，而且考虑到Etsy当时的规模，维护成本开始随着我们必须...