Of apples and oranges, file drawers and garbage: why validity issues in meta-analysis will not go away

Abstract

This paper examines how threats to the validity of meta-analysis have been dealt with by clinical researchers employing this approach to literature review. Three validity threats were identified--mixing of dissimilar studies, publication bias, and inclusion of poor quality studies. Approaches to addressing these threats were evaluated for their effectiveness and popularity by surveying 32 published meta-analyses in clinical psychology. Distrust of meta-analysis, however, was found to transcend these validity threats. Other explanations for why this popular research strategy continues to receive widespread criticism were considered. Suggestions were made for how meta-analysis might better address these concerns.