Statistics - Effect Size

Thomas Bayes


Effect size tells you about the magnitude of the effect.


Glass coined the term “meta-analysis”. 1978: Gene V. Glass statistically aggregate the findings of 375 psychotherapy outcome studies Glass (and colleague Smith) to disprove claim that psychotherapy was useless.

Effect Size is used prolifically in meta-analysis to combine results from multiple studies (averaging results from different experiments can produce nonsense).

“When a number of quite independent tests of significance have been made, it sometimes happens that although few or none can be claimed individually as significant, yet the aggregate gives an impression that the probabilities are on the whole lower than would often have been obtained by chance”

Fisher (1944)


<MATH> \text{Effect Size} = \frac{ \displaystyle \text{Mean of experimental group} – \text{Mean of control group} } { \displaystyle \href{Standard deviation}{\text{Standard deviation}}} </MATH>

Other definitions of effect size exist: odds-­ratio, correlation coefficient

Cohen’s Heuristic: Standardized mean difference effect size

  • small = 0.20
  • medium = 0.50
  • large = 0.80


Weighted Average

  • Idea: Average across multiple studies, but give more weight to more precise studies
  • Simple method: Weight by sample size: w_i = {n_i}/{Sigma_j n_j}
  • Inverse-variance weight: w_i = 1/{se^2}

Standardized Mean Difference

Efeect Sized (ES) = {{Mean of group1} - {Mean of group2}}/{Standard deviation pooled} = {overline{X}_1 - overline{X}_2}/{sigma_{pooled}}

Different Definitions of {sigma_{pooled}}

  • Glass 1976 = {sigma_{pooled}} = hat{{sigma}_2}
  • Hartung et al., 2008 …

Confidence Interval

Confidence Interval (of effect size)

A 95% confidence interval of the effect size mean that if the experiment 100 times is repeated 100 times, we expect that the interval would include this effect size 95/100 times

If this interval includes 0.0, that’s equivalent to saying the result is not statistically significant.

Discover More
Thomas Bayes
Statistics - (Student's) t-test (Mean Comparison)

The t-test is a test that compares means. NHST can be conducted yielding to a p-value Effect Size can be calculated like in multiple regression. Confidence Interval around the mean can also be...
Thomas Bayes
Statistics - (dependent|paired sample) t-test

A dependent t-test is appropriate when: we have the same people measured twice. the same subject are been compared (ex: Pre/Post Design) or two samples are matched at the level of individual subjects...
Thomas Bayes
Statistics - Effects (between predictor variable)

Effect between predictor variable interaction simple main See also:
Card Puncher Data Processing
Statistics - Null Hypothesis Significance Testing (NHST)

NHST is a procedure for testing the Null Hypothesis. It's a binary decision: Reject or Retain the Null Hypothesis Ie Retain the Null Hypothesis or the alternative hypothesis. Before starting any...
Thomas Bayes
Statistics - independent t-test

A dependent t-test is appropriate when you want to compare two independent samples. (two completely different groups of entities). For examples: if we want to compare one condition to another as men...

Share this page:
Follow us:
Task Runner