Research methods: managing primary study quality in meta-analyses

Abstract

Meta-analyses synthesize multiple primary studies and identify patterns of relationships. Differences in primary study methodological quality must be addressed for meta-analysis to produce meaningful results. No single standard exists for addressing these quality variations. Quality measurement scales are fraught with development and application problems. Several strategies have been proposed to address quality. Researchers can set minimum levels for inclusion or require that certain quality attributes be present. An inclusive method is to weight effect sizes by quality scores. This allows the inclusion of diverse studies but relies on questionable quality measures. By considering quality an empirical question, meta-analysts can examine associations between quality and effect sizes and thus preserve the purpose of meta-analysis to systematically examine data. Researchers increasingly are combining strategies to overcome the limitations of using a single approach. Future work to develop valid measures of primary study quality dimensions will improve the ability of meta-analysis to inform research and nursing practice.