解读A/B测试结果：假阴性和力量

Martin Tingley with Wenjing Zheng, Simon Ejdemyr, Stephanie Lane, and Colin McFarland

马丁-丁利 与 郑文静, Simon Ejdemyr, 斯蒂芬妮-莱恩, 和 科林-麦克法兰

This is the fourth post in a multi-part series on how Netflix uses A/B tests to inform decisions and continuously innovate on our products. Need to catch up? Have a look at Part 1 (Decision Making at Netflix), Part 2 (What is an A/B Test?), Part 3 (False positives and statistical significance). Subsequent posts will go into more details on experimentation across Netflix, how Netflix has invested in infrastructure to support and scale experimentation, and the importance of the culture of experimentation within Netflix.

这是关于Netflix如何使用A/B测试为决策提供信息并不断创新产品的多部分系列的第四篇文章。需要了解一下吗？请看 第一部分 (Netflix的决策)。 第二部分 (什么是A/B测试？)。 第三部分 (假阳性和统计意义）。)随后的文章将更详细地介绍整个Netflix的实验，Netflix如何投资基础设施以支持和扩大实验，以及Netflix内部实验文化的重要性。

In Part 3: False positives and statistical significance, we defined the two types of mistakes that can occur when interpreting test results: false positives and false negatives. We then used simple thought exercises based on flipping coins to build intuition around false positives and related concepts such as statistical significance, p-values, and confidence intervals. In this post, we’ll do the same for false negatives and the related concept of statistical power.

在 第三部分：假阳性和统计意义中，我们定义了解释测试结果时可能发生的两种错误：假阳性和假阴性。然后，我们利用基于抛硬币的简单思维练习，围绕假阳性和相关概念（如统计学意义、P值和置信区间）建立直觉。在这篇文章中，我们将对假阴性和相关的统计能力概念做同样的练习。

Figure 1: As in Part 3, we’ll use thought exercises based on flipping coins, such as this one displaying Caesar Augustus, to build up intuition about core statistical concepts.

图1：和第三部分一样，我们将使用基于投掷硬币的思维练习，比如这个显示凯撒-奥古斯都的练习，来建立对核心统计概念的直觉。

False negatives and power

错误的否定和权力

A false negative occurs when the data do not indicate a meaningful difference between treatment and control, but in truth there is a difference. Continuing on an example from Part 3, a false negative corresponds to labeling the photo of the cat as a “not cat.” False negatives are closely related to the statistical concept of power, whi...