在功能推出过程中自动进行多臂盗贼测试
A/B testing is an experiment where a random e-commerce platform user is given two versions of a variable: a control group and a treatment group, to discover the optimal version that maximizes conversion. When running A/B testing, you can take the Multi-Armed Bandit optimisation approach to minimise the loss of conversion due to low performance.
A/B测试是一个实验，随机给一个电子商务平台用户提供一个变量的两个版本：一个对照组和一个处理组，以发现使转换率最大化的最佳版本。在进行A/B测试时，你可以采取多臂强盗的优化方法，以尽量减少因性能低下而造成的转换损失。
In the traditional software development process, Multi-Armed Bandit (MAB) testing and rolling out a new feature are usually separate processes. The novel Multi-Armed Bandit System for Recommendation solution, hereafter the Multi-Armed Bandit Optimiser, proposes automating the Multi-Armed Bandit testing simultaneously while rolling out the new feature.
在传统的软件开发过程中，多臂匪徒（MAB）测试和推出新功能通常是独立的过程。新颖的多臂强盗系统推荐方案，以下简称多臂强盗优化器，提出在推出新功能的同时自动进行多臂强盗测试。
Advantages
优势
- Automates the MAB testing process during new feature rollouts.
- 在新功能推出期间，自动进行MAB测试过程。
- Selects the optimal parameters based on predefined metrics of each use case, which results in an end-to-end solution without the need for user intervention.
- 根据每个用例的预定义指标选择最佳参数，从而形成端到端的解决方案，无需用户干预。
- Uses the Batched Multi-Armed Bandit and Monte Carlo Simulation, which enables it to process large-scale business scenarios.
- 使用Batched Multi-Armed Bandit和Monte Carlo Simulation，这使得它能够处理大规模的商业场景。
- Uses a feedback loop to automatically collect recommendation metrics from user event logs and to feed them to the Multi-Armed Bandit Optimiser.
- 使用一个反馈回路，从用户事件日志中自动收集推荐指标，并将其反馈给多臂强盗优化器。
- Uses an adaptive rollout method to automatically roll out the best model to the maximum distribution capacity according to the feedback metrics.
- 使用自适应推出方法，根据反馈指标，自动将最佳模型推出到最大分配能力。
Architecture
架构
The following diagram illustrates the system architecture.
下图说明了系统结构。
System architecture
The novel Multi-Armed Bandit System for Recommendation solution contains three building blocks.
新颖的多臂强盗系统的推荐方案包含三个构件。
- Stream processing framework
- 流处理框架
A lightweight system that performs basic o...