抓取实验决策引擎 - 一个统一的实验工具包

This article introduces the GrabX Decision Engine, an internal open-source package that offers a comprehensive framework for designing and analysing experiments conducted on online experiment platforms. The package encompasses a wide range of functionalities, including a pre-experiment advisor, a post-experiment analysis toolbox, and other advanced tools. In this article, we explore the motivation behind the development of these functionalities, their integration into the unique ecosystem of Grab’s multi-sided marketplace, and how these solutions strengthen the culture and calibre of experimentation at Grab.

本文介绍了GrabX决策引擎，这是一个内部开源软件包，提供了一个全面的框架，用于设计和分析在在线实验平台上进行的实验。该软件包包含了各种功能，包括预实验顾问、后实验分析工具箱和其他高级工具。在本文中，我们探讨了开发这些功能的动机，它们如何融入Grab独特的多边市场生态系统，并如何加强Grab实验文化和实验质量。

Background

背景

Today, Grab’s Experimentation (GrabX) platform orchestrates the testing of thousands of experimental variants each week. As the platform continues to expand and manage a growing volume of experiments, the need for dependable, scalable, and trustworthy experimentation tools becomes increasingly critical for data-driven and evidence-based decision-making.

如今，Grab的实验平台（GrabX）每周协调测试数千个实验变体。随着平台的不断扩大和管理越来越多的实验，对可靠、可扩展和值得信赖的实验工具的需求对于基于数据和证据的决策变得越来越关键。

In our previous article, we presented the Automated Experiment Analysis application, a tool designed to automate data pipelines for analyses. However, during the development of this application for Grab’s experimenter community, we noticed a prevailing trend: experiments were predominantly analysed on a one-by-one, manual basis. While such a federated approach may be needed in a few cases, it presents numerous challenges at the organisational level:

在我们之前的文章中，我们介绍了Automated Experiment Analysis应用程序，这是一个旨在自动化数据分析流程的工具。然而，在为Grab的实验者社区开发这个应用程序的过程中，我们注意到了一个普遍趋势：实验主要是以逐个手动分析的方式进行的。虽然在一些情况下可能需要这种联邦方法，但它在组织层面上存在许多挑战：

• Lack of a contextual toolkit: GrabX facilitates executing a diverse range of experimentation designs, catering to the varied needs and contexts of different tech teams across the organisation. ...