Table of Contents

Statistics - NHST Errors

Error

Type I

Probability of Type I errors increases when researchers conduct multiple NHSTs.

A Type I error means that the null hypothesis is true, but you reject the null hypothesis.

If you set your p-level for significance at 0.05, that means that if the null hypothesis is true then you have a 5% chance of getting data as extreme or more so than the observed data, and thus a 5% chance of rejecting the null hypothesis even though the null hypothesis is true.

If you repeat the pairwise test for different pairs, that 5% chance for each pairwise test accumulates.

For example, if you do 14 pairwise tests using p = 0.05 as the threshold for significance, then the chance of making a Type I error is : <math>1 - 0.95^{14}= 51 \% </math>

because the probability of several independent events all occurring is the product of their individual probabilities.

Type II

Many fields of research are plagued by a large degree of sampling error, which makes it difficult to detect an effect, even when the effect exists