Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2024 Oct;66(7):e202300101.
doi: 10.1002/bimj.202300101.

Meta-Analysis of Diagnostic Accuracy Studies With Multiple Thresholds: Comparison of Approaches in a Simulation Study

Antonia Zapf  1 , Cornelia Frömke  2 , Juliane Hardt  3 , Gerta Rücker  4 , Dina Voeltz  5   6 , Annika Hoyer  5
Affiliations
Comparative Study

Meta-Analysis of Diagnostic Accuracy Studies With Multiple Thresholds: Comparison of Approaches in a Simulation Study

Antonia Zapf et al. Biom J. 2024 Oct.

Abstract

The development of methods for the meta-analysis of diagnostic test accuracy (DTA) studies is still an active area of research. While methods for the standard case where each study reports a single pair of sensitivity and specificity are nearly routinely applied nowadays, methods to meta-analyze receiver operating characteristic (ROC) curves are not widely used. This situation is more complex, as each primary DTA study may report on several pairs of sensitivity and specificity, each corresponding to a different threshold. In a case study published earlier, we applied a number of methods for meta-analyzing DTA studies with multiple thresholds to a real-world data example (Zapf et al., Biometrical Journal. 2021; 63(4): 699-711). To date, no simulation study exists that systematically compares different approaches with respect to their performance in various scenarios when the truth is known. In this article, we aim to fill this gap and present the results of a simulation study that compares three frequentist approaches for the meta-analysis of ROC curves. We performed a systematic simulation study, motivated by an example from medical research. In the simulations, all three approaches worked partially well. The approach by Hoyer and colleagues was slightly superior in most scenarios and is recommended in practice.

Keywords: diagnostic test accuracy studies; meta‐analysis; receiver operating characteristic curve; simulation study.

PubMed Disclaimer

References

    1. Albert, C., A. Albert, R. Bellomo, et al. 2018. “Urinary Neutrophil Gelatinase‐Associated Lipocalin‐Guided Risk Assessment for Major Adverse Kidney Events After Open‐Heart Surgery.” Biomarkers Medicine 12: 975–985. https://doi.org/10.2217/bmm‐2018‐0071.
    1. Arends, L. R., T. H. Hamza, J. C. van Houwelingen, M. H. Heijenbrok‐Kal, M. G. Hunink, and T. Stijnen. 2008. “Bivariate Random Effects Meta‐Analysis of ROC Curves.” Medical Decision Making 28, no. 5: 621–638. https://doi.org/10.1177/0272989X08319957.
    1. Benedetti, A., B. Levis, G. Rücker, et al., DEPRESsion Screening Data (DEPRESSD) Collaboration. 2020. “An Empirical Comparison of Three Methods for Multiple Cutoff Diagnostic Test Meta‐Analysis of the Patient Health Questionnaire‐9 (PHQ‐9) Depression Screening Tool Using Published Data vs. Individual Level Data.” Research Synthesis Methods 11, no. 6: 833–848. https://doi.org/10.1002/jrsm.1443.
    1. Boulesteix, A. L., R. Wilson, and A. Hapfelmeier. 2017. “Towards Evidence‐Based Computational Statistics: Lessons From Clinical Research on the Role and Design of Real‐Data Benchmark Studies.” BMC Medical Research Methodology 17, no. 1: 1–12. https://doi.org/10.1186/s12874‐017‐0417‐2.
    1. Burton, A., D. G. Altman, P. Royston, and R. L. Holder. 2006. “The Design of Simulation Studies in Medical Statistics.” Statistics in Medicine 25, no. 24: 4279–4292. https://doi.org/10.1002/sim.2673.

Publication types

LinkOut - more resources