What if studies aren’t identical?


If you had data that fit the common effect model, you’d just weight each study result by that study’s precision. You’d put all of those weighted results together, average them, and have an answer to your question.

However, we know that in most cases, there are various factors that make study results different. Humans and human conditions aren’t as predictable as a fair coin. The effect of a treatment for a woman in Alaska is unlikely to be the same as the effect for a man in Kenya.

That means that we have to consider that the truth for one study might not be the same as the truth for a study that happened in a different place with different types of people. In other words, study findings are subject to two sources of variation –

1. Variation between the “true effect” in one study and the observed results for that study
2. Variation between the “true effect” in one study and the “true effect” in another study.

To take into account these two types of variation, we use what’s called the “random effects model.”