什么是A/B测试？
Martin Tingley with Wenjing Zheng, Simon Ejdemyr, Stephanie Lane, and Colin McFarland
马丁-丁利 与 郑文静, Simon Ejdemyr, 斯蒂芬妮-莱恩, 和 科林-麦克法兰
This is the second post in a multi-part series on how Netflix uses A/B tests to inform decisions and continuously innovate on our products. See here for Part 1: Decision Making at Netflix. Subsequent posts will go into more details on the statistics of A/B tests, experimentation across Netflix, how Netflix has invested in infrastructure to support and scale experimentation, and the importance of the culture of experimentation within Netflix.
这是关于Netflix如何使用A/B测试为决策提供信息并不断对我们的产品进行创新的多部分系列中的第二篇文章。请看 这里 第一部分。Netflix的决策。随后的文章将详细介绍A/B测试的统计数据、Netflix的实验、Netflix如何投资基础设施以支持和扩大实验，以及Netflix内部实验文化的重要性。
An A/B test is a simple controlled experiment. Let’s say — this is a hypothetical! — we want to learn if a new product experience that flips all of the boxart upside down in the TV UI is good for our members.
A/B测试是一个简单的受控实验。比方说--这只是一个假设！我们想了解一个新的产品体验，即在电视用户界面上把所有的盒子都翻过来，对我们的会员是否有好处。- 我们想了解一个新的产品体验，即在电视用户界面上将所有的盒子翻转过来，是否对我们的会员有好处。
Figure 1: How do we decide if Product Experience B, with the Upside Down box art, is a better experience for our members?
图1：我们如何决定产品体验B，即颠倒的盒子设计，对我们的会员来说是否是更好的体验？
To run the experiment, we take a subset of our members, usually a simple random sample, and then use random assignment to evenly split that sample into two groups. Group “A,” often called the “control group,” continues to receive the base Netflix UI experience, while Group “B,” often called the “treatment group”, receives a different experience, based on a specific hypothesis about improving the member experience (more on those hypotheses below). Here, Group B receives the Upside Down box art.
为了进行实验，我们从我们的会员中抽取一个子集，通常是一个简单的随机样本，然后使用随机分配将该样本平均分成两组。A组"，通常被称为 "对照组"，继续接受基本的Netflix用户界面体验，而B组，通常被称为 "治疗组"，接受不同的体验，基于关于改善会员体验的特定假设（关于这些假设的详情见下文）。在这里，B组接受的是颠倒的盒子设计。
We wait, and we then compare the values of a variety of metrics from Group A to those from Group B. Some metrics will be specific to the given hypothesis. F...