Common Mistakes in Meta-Analysis and How to Avoid Them

Part 2: How to choose between the fixed-effect model and the random-effects model

Abstract

There are two popular statistical models for meta-analysis, the fixed-effect model and the random-effects model.

Under the fixed-effect model we assume that there is one true effect size that underlies all the studies in the analysis, and that all differences in observed effects are due to sampling error. While we follow the practice of calling this a fixed-effect model, a more descriptive term would be a common-effect model. In either case, we use the singular (effect) since there is only one true effect.

By contrast, under the random-effects model we allow that the true effect size might differ from study to study. For example, the effect size might be higher (or lower) in studies where the participants are older, or more educated, or healthier than in other studies, or when a more intensive variant of an intervention is used. The term "Random" reflects the fact that the studies included in the analysis are assumed to be a random sample of all possible studies that meet the inclusion criteria for the review. And we use the plural (effects) since we are working with multiple true effects.

Download Part 2 as a PDF


Part 1: I-squared is not an absolute measure of heterogeneity in a meta-analysis

Abstract

Researchers often use the I2 index to quantify the dispersion of effect sizes in a meta-analysis. Some suggest that I2 values of 25%, 50%, and 75%, correspond to small, moderate, and large amounts of heterogeneity. In fact though, I2 is a not a measure of absolute heterogeneity. Rather, it tells us what proportion of the observed variance reflects variance in true effect sizes rather than sampling error. This distinction between an absolute number and a proportion is fundamental to the correct interpretation of I2. A meta-analysis with a low value of I2 could have only trivial heterogeneity but could also have substantial heterogeneity. Conversely, a meta-analysis with a high value of I2 could have substantial heterogeneity, but could also have only trivial heterogeneity. Our goal in this paper is to explain what I2 is, and how it should (and should not) be used in meta-analysis.

Download Part 1 as a PDF

researchers
Testimonials

"We very much enjoyed and benefitted from your expertise. Students and faculty alike in public health and psychology have been motivated to commence meta-analyses research because of this workshop."

Dr. R. Scott Olds - Professor and Interim Chair, Department of Social and Behavioral Sciences, College of Public Health, Kent State University


"I was interested in learning how to do a systematic review and meta-analysis. I´ve done several courses but couldn't evolve to a complete analysis due to the limitations of available software. Then I became aware of "Comprehensive Meta-Analysis". The program is very intuitive and the book "Introduction to Meta-Analysis" is the most objective I've ever read in this topic. I have recommended to those interested in starting training in systematic review to begin by this book and software."

Ricardo Botelho, MD, Ph.D - Post-graduate course - IAMSPE, São Paulo, Brazil