Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Sep 27;24(1):217.
doi: 10.1186/s12874-024-02321-3.

Implementing multiple imputations for addressing missing data in multireader multicase design studies

Zhemin Pan #  1 , Yingyi Qin #  2 , Wangyang Bai  1 , Qian He  2 , Xiaoping Yin  3 , Jia He  4   5
Affiliations

Implementing multiple imputations for addressing missing data in multireader multicase design studies

Zhemin Pan et al. BMC Med Res Methodol. .

Abstract

Background: In computer-aided diagnosis (CAD) studies utilizing multireader multicase (MRMC) designs, missing data might occur when there are instances of misinterpretation or oversight by the reader or problems with measurement techniques. Improper handling of these missing data can lead to bias. However, little research has been conducted on addressing the missing data issue within the MRMC framework.

Methods: We introduced a novel approach that integrates multiple imputation with MRMC analysis (MI-MRMC). An elaborate simulation study was conducted to compare the efficacy of our proposed approach with that of the traditional complete case analysis strategy within the MRMC design. Furthermore, we applied these approaches to a real MRMC design CAD study on aneurysm detection via head and neck CT angiograms to further validate their practicality.

Results: Compared with traditional complete case analysis, the simulation study demonstrated the MI-MRMC approach provides an almost unbiased estimate of diagnostic capability, alongside satisfactory performance in terms of statistical power and the type I error rate within the MRMC framework, even in small sample scenarios. In the real CAD study, the proposed MI-MRMC method further demonstrated strong performance in terms of both point estimates and confidence intervals compared with traditional complete case analysis.

Conclusion: Within MRMC design settings, the adoption of an MI-MRMC approach in the face of missing data can facilitate the attainment of unbiased and robust estimates of diagnostic capability.

Keywords: Computer-aided diagnosis; Missing data; Multiple imputation; Multireader multicase.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Mean type I error performance under different scenarios differentiated by sample size for the original, CC and MI-MRMC approaches. A) Under the MCAR mechanism, B) under the MAR mechanism. CC: complete case analysis, MI-MRMC: multiple imputation under the MRMC framework, Original: DBM analysis on the original complete dataset
Fig. 2
Fig. 2
Mean power performance under different scenarios differentiated by sample size for the original, CC and MI-MRMC approaches. A) Under the MCAR mechanism, B) under the MAR mechanism. CC: complete case analysis, MI-MRMC: multiple imputation under the MRMC framework, Original: DBM analysis on the original complete dataset
Fig. 3
Fig. 3
Mean RMSE performance under different scenarios for the original, CC analysis and MI-MRMC approaches. CC: complete case analysis, MI-MRMC: multiple imputation under the MRMC framework, Original: DBM analysis on the original complete dataset, NA: not applicable
Fig. 4
Fig. 4
Mean bias performance under different scenarios for the original, CC analysis and MI-MRMC approaches. CC: complete case analysis, MI-MRMC: multiple imputation under the MRMC framework, Original: DBM analysis on the original complete dataset, NA: not applicable
Fig. 5
Fig. 5
Mean 95% confidence interval coverage rate performance under different scenarios for the original, CC analysis and MI-MRMC approaches CC: complete case analysis, MI-MRMC: multiple imputation under the MRMC framework, Original: DBM analysis on the original complete dataset, NA: not applicable
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Gallas BD, Chan HP, D’Orsi CJ, Dodd LE, Giger ML, Gur D, Krupinski EA, Metz CE, Myers KJ, Obuchowski NA, et al. Evaluating imaging and computer-aided detection and diagnosis devices at the FDA. Acad Radiol. 2012;19(4):463–77. - PMC - PubMed
    1. Wagner RF, Metz CE, Campbell G. Assessment of medical imaging systems and computer aids: a tutorial review. Acad Radiol. 2007;14(6):723–48. - PubMed
    1. Beam CA, Layde PM, Sullivan DC. Variability in the interpretation of screening mammograms by US radiologists. Findings from a national sample. Arch Intern Med. 1996;156(2):209–13. - PubMed
    1. Yu T, Li Q, Gray G, Yue LQ. Statistical innovations in diagnostic device evaluation. J Biopharm Stat. 2016;26(6):1067–77. - PubMed
    1. Clinical Performance Assessment. Considerations for Computer-Assisted Detection Devices Applied to Radiology Images and Radiology Device Data in Premarket Notification (510(k)) Submissions: Guidance for Industry and Food and Drug Administration Staff [https://www.fda.gov/media/77642/download].

LinkOut - more resources