Combining studies using effect sizes and quality scores: application to bone loss in postmenopausal women

Abstract

This article presents a random effects model that uses effect sizes (ES) and quality scores to integrate results from investigations. An empirical example is given with data obtained from a meta-analysis on the effectiveness of physical activity in the prevention of bone loss in healthy postmenopausal women. A Medline search was performed to locate relevant studies published in French or English between January 1966 and May 1996. Three independent reviewers extracted data from studies. Effect sizes were calculated according to the method of Hedges and Olkin. A modified version of Chalmers' scale was utilized to calculate quality scores. DerSimonian and Laird's method with incorporation of the quality scores was used to estimate the overall effect size. Quality scores and the inverse of the variances were included as weights when combining studies. The overall estimate and standard error (SE) of the effect of physical activity on spinal bone mineral density loss in healthy postmenopausal women was ESoverall = 0.4263 (1.1361). When compared to other meta-analysis methods such as the fixed effects model and the model of DerSimonian and Laird without the quality score (DL), the new model generated comparable estimators (fixed effects model's ESoverall (SE) = 1.2724 (0.0139), DLs ESoverall (SE) = 0.3958 (1.2370)). Due to the heterogeneity that existed between studies, a random effects model was more appropriate then a fixed effects model. However, it resulted in wider confidence intervals, as expected. It was shown empirically that the model using quality scores generated narrower confidence intervals than the model of DL alone. The inclusion of covariates such as quality scores in meta-analyses permits the quantification of the variation between studies.