Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Apr 16;20(1):85.
doi: 10.1186/s12874-020-00952-w.

Data sources and methods used to determine pretest probabilities in a cohort of Cochrane diagnostic test accuracy reviews

Michiel S Oerbekke  1   2 , Kevin Jenniskens  3 , Rob J P M Scholten  4   3 , Lotty Hooft  4   3
Affiliations

Data sources and methods used to determine pretest probabilities in a cohort of Cochrane diagnostic test accuracy reviews

Michiel S Oerbekke et al. BMC Med Res Methodol. .

Abstract

Background: A pretest probability must be selected to calculate data to help clinicians, guideline boards and policy makers interpret diagnostic accuracy parameters. When multiple analyses for the same target condition are compared, identical pretest probabilities might be selected to facilitate the comparison. Some pretest probabilities may lead to exaggerations of the patient harms or benefits, and guidance on how and why to select a specific pretest probability is minimally described. Therefore, the aim of this study was to assess the data sources and methods used in Cochrane diagnostic test accuracy (DTA) reviews for determining pretest probabilities to facilitate the interpretation of DTA parameters. A secondary aim was to assess the use of identical pretest probabilities to compare multiple meta-analyses within the same target condition.

Methods: Cochrane DTA reviews presenting at least one meta-analytic estimate of the sensitivity and/or specificity as a primary analysis published between 2008 and January 2018 were included. Study selection and data extraction were performed by one author and checked by other authors. Observed data sources (e.g. studies in the review, or external sources) and methods to select pretest probabilities (e.g. median) were categorized.

Results: Fifty-nine DTA reviews were included, comprising of 308 meta-analyses. A pretest probability was used in 148 analyses. Authors used included studies in the DTA review, external sources, and author consensus as data sources for the pretest probability. Measures of central tendency with or without a measure of dispersion were used to determine the pretest probabilities, with the median most commonly used. Thirty-two target conditions had at least one identical pretest probability for all of the meta-analyses within their target condition. About half of the used identical pretest probabilities were inside the prevalence ranges from all analyses within a target condition.

Conclusions: Multiple sources and methods were used to determine (identical) pretest probabilities in Cochrane DTA reviews. Indirectness and severity of downstream consequences may influence the acceptability of the certainty in calculated data with pretest probabilities. Consider: whether to present normalized frequencies, the influence of pretest probabilities on normalized frequencies, and whether to use identical pretest probabilities for meta-analyses in a target condition.

Keywords: Absolute numbers; Background prevalence; Diagnostic test accuracy; Natural frequencies; Normalized frequencies; Pretest probability; Pretest risk; Review literature as topic.

PubMed Disclaimer

Conflict of interest statement

Authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Cohort formation. Flow diagram showing the formation of the cohort and reasons for exclusion. CDSR: Cochrane Database of Systematic Reviews
Fig. 2
Fig. 2
Data sources and methods for determining pretest probabilities in Cochrane Diagnostic Test Accuracy reviews. Sankey plot showing which data sources and methods were used to determine the pretest probability in Cochrane Diagnostic Test Accuracy reviews. IQR: Interquartile Range, WHO: World Health Organization
Fig. 3
Fig. 3
Identical pretest probabilities within target conditions. The figure shows the number of target conditions with two or more analyses and the use of identical pretest probabilities for all of the meta-analyses within a target condition. Black bars indicate the number of pretest probabilities that were outside the prevalence range of at least one meta-analysis for the target condition. White bars indicate the number of pretest probabilities that were inside the prevalence ranges of all individual meta-analyses for that specific target condition. The gray bar indicates that it was unclear whether these identical pretest probabilities were inside or outside the disease prevalence ranges. No bars were drawn in the figure when there were no identical pretest probabilities used for all of the meta-analyses within a target condition. DTA: Diagnostic Test Accuracy, TC: Target Condition
See this image and copyright information in PMC

References

    1. Ferrante di Ruffano L, Dinnes J, Sitch AJ, Hyde C, Deeks JJ. Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests. BMC Med Res Methodol. 2017;17(1):35. doi: 10.1186/s12874-016-0287-z. - DOI - PMC - PubMed
    1. Whiting PF, Davenport C, Jameson C, Burke M, Sterne JA, Hyde C, Ben-Shlomo Y. How well do health professionals interpret diagnostic information? A systematic review. BMJ Open. 2015;5(7):e008155. doi: 10.1136/bmjopen-2015-008155. - DOI - PMC - PubMed
    1. Mustafa RA, Wiercioch W, Santesso N, Cheung A, Prediger B, Baldeh T, Carrasco-Labra A, Brignardello-Petersen R, Neumann I, Bossuyt P, et al. Decision-making about healthcare related tests and diagnostic strategies: user testing of GRADE evidence tables. PLoS One. 2015;10(10):e0134553. doi: 10.1371/journal.pone.0134553. - DOI - PMC - PubMed
    1. Schüneman H, Brożek J, Guyatt G, Oxman A, editors: Chapter 7: The GRADE Approach for Diagnostic Tests and Strategies. In: Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach, Updated October 2013. edn.; Updated October 2013. Available from https://guidelinedevelopment.org/handbook.
    1. Bossuyt P, Davenport C, Deeks J, Hyde C, Leeflang M, Scholten R: Chapter 11: Interpreting results and drawing conclusions. In: Cochrane Handbook for Systematic Reviews of Diagostic Test Accuracy, Version 09. edn. Edited by Deeks J, Bossuyt P, Gatsonis C; 2013, Available from: http://methods.cochrane.org/sdt/handbook-dta-reviews.