Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2005 May 26:5:19.
doi: 10.1186/1471-2288-5-19.

No role for quality scores in systematic reviews of diagnostic accuracy studies

Penny Whiting  1 , Roger Harbord, Jos Kleijnen
Affiliations

No role for quality scores in systematic reviews of diagnostic accuracy studies

Penny Whiting et al. BMC Med Res Methodol. .

Abstract

Background: There is a lack of consensus regarding the use of quality scores in diagnostic systematic reviews. The objective of this study was to use different methods of weighting items included in a quality assessment tool for diagnostic accuracy studies (QUADAS) to produce an overall quality score, and to examine the effects of incorporating these into a systematic review.

Methods: We developed five schemes for weighting QUADAS to produce quality scores. We used three methods to investigate the effects of quality scores on test performance. We used a set of 28 studies that assessed the accuracy of ultrasound for the diagnosis of vesico-ureteral reflux in children.

Results: The different methods of weighting individual items from the same quality assessment tool produced different quality scores. The different scoring schemes ranked different studies in different orders; this was especially evident for the intermediate quality studies. Comparing the results of studies stratified as "high" and "low" quality based on quality scores resulted in different conclusions regarding the effects of quality on estimates of diagnostic accuracy depending on the method used to produce the quality score. A similar effect was observed when quality scores were included in meta-regression analysis as continuous variables, although the differences were less apparent.

Conclusion: Quality scores should not be incorporated into diagnostic systematic reviews. Incorporation of the results of the quality assessment into the systematic review should involve investigation of the association of individual quality items with estimates of diagnostic accuracy, rather than using a combined quality score.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Estimates of sensitivity and 1-specificity plotted in ROC space for standard and contrast enhanced ultrasound
Figure 2
Figure 2
ranking of studies according to each different quality score
Figure 3
Figure 3
Forest plots showing the RDOR in "high" quality studies compared to "low" quality studies for each of the five quality scoring schemes
Figure 4
Figure 4
Forest plots showing the RDOR for a 10 point increase in quality for each of the 5 quality scoring schemes
See this image and copyright information in PMC

References

    1. Juni P, Altman DG, Egger M. Assessing the quality of controlled trials. BMJ. 2001;323:42–46. doi: 10.1136/bmj.323.7303.42. - DOI - PMC - PubMed
    1. Assendelft JJ, Koes BW, van Tulder MW, Bouter LM. Scoring the quality of clinical trials [letter] JAMA. 2000;283:1421. doi: 10.1001/jama.283.11.1421. - DOI - PubMed
    1. ter Riet G, Leffers P, Zeegers M. Scoring the quality of clinical trials [letter]. JAMA. 2000;283:1421. doi: 10.1001/jama.283.11.1421. - DOI - PubMed
    1. Berlin JA, Rennie D. Measuring the quality of trials: the quality of quality scales. JAMA. 1999;282:1083–5. doi: 10.1001/jama.282.11.1083. - DOI - PubMed
    1. Juni P, Egger M. Scoring the quality of clinical trials [letter] JAMA. 2000;283:1422–3. - PubMed

Publication types

LinkOut - more resources