Multireader multicase variance analysis for binary data
- PMID: 18059916
- DOI: 10.1364/josaa.24.000b70
Multireader multicase variance analysis for binary data
Abstract
Multireader multicase (MRMC) variance analysis has become widely utilized to analyze observer studies for which the summary measure is the area under the receiver operating characteristic (ROC) curve. We extend MRMC variance analysis to binary data and also to generic study designs in which every reader may not interpret every case. A subset of the fundamental moments central to MRMC variance analysis of the area under the ROC curve (AUC) is found to be required. Through multiple simulation configurations, we compare our unbiased variance estimates to naïve estimates across a range of study designs, average percent correct, and numbers of readers and cases.
Similar articles
-
The average receiver operating characteristic curve in multireader multicase imaging studies.Br J Radiol. 2014 Aug;87(1040):20140016. doi: 10.1259/bjr.20140016. Epub 2014 Jun 2. Br J Radiol. 2014. PMID: 24884728 Free PMC article.
-
One-shot estimate of MRMC variance: AUC.Acad Radiol. 2006 Mar;13(3):353-62. doi: 10.1016/j.acra.2005.11.030. Acad Radiol. 2006. PMID: 16488848
-
Multireader multicase reader studies with binary agreement data: simulation, analysis, validation, and sizing.J Med Imaging (Bellingham). 2014 Oct;1(3):031011. doi: 10.1117/1.JMI.1.3.031011. Epub 2014 Dec 4. J Med Imaging (Bellingham). 2014. PMID: 26158051 Free PMC article.
-
Multireader, multicase receiver operating characteristic analysis: an empirical comparison of five methods.Acad Radiol. 2004 Sep;11(9):980-95. doi: 10.1016/j.acra.2004.04.014. Acad Radiol. 2004. PMID: 15350579
-
Multireader Diagnostic Accuracy Imaging Studies: Fundamentals of Design and Analysis.Radiology. 2022 Apr;303(1):26-34. doi: 10.1148/radiol.211593. Epub 2022 Feb 15. Radiology. 2022. PMID: 35166584 Review.
Cited by
-
Multi-reader ROC studies with split-plot designs: a comparison of statistical methods.Acad Radiol. 2012 Dec;19(12):1508-17. doi: 10.1016/j.acra.2012.09.012. Acad Radiol. 2012. PMID: 23122570 Free PMC article.
-
Evaluation of computer-aided detection and diagnosis systems.Med Phys. 2013 Aug;40(8):087001. doi: 10.1118/1.4816310. Med Phys. 2013. PMID: 23927365 Free PMC article.
-
Regulatory considerations for medical imaging AI/ML devices in the United States: concepts and challenges.J Med Imaging (Bellingham). 2023 Sep;10(5):051804. doi: 10.1117/1.JMI.10.5.051804. Epub 2023 Jun 23. J Med Imaging (Bellingham). 2023. PMID: 37361549 Free PMC article. Review.
-
Optimizing data acquisition in undersampled magnetic resonance imaging (MRI) using two alternative forced choice (2-AFC) and search tasks.Proc SPIE Int Soc Opt Eng. 2023 Feb;12467:124670U. doi: 10.1117/12.2654323. Epub 2023 Apr 3. Proc SPIE Int Soc Opt Eng. 2023. PMID: 37131343 Free PMC article.
-
Utility of a Deep-Learning Algorithm to Guide Novices to Acquire Echocardiograms for Limited Diagnostic Use.JAMA Cardiol. 2021 Jun 1;6(6):624-632. doi: 10.1001/jamacardio.2021.0185. JAMA Cardiol. 2021. PMID: 33599681 Free PMC article.
LinkOut - more resources
Full Text Sources