Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Sep 3;14(9):e0221339.
doi: 10.1371/journal.pone.0221339. eCollection 2019.

A systematic review of the diagnostic accuracy of artificial intelligence-based computer programs to analyze chest x-rays for pulmonary tuberculosis

Miriam Harris  1   2   3 , Amy Qi  2   4 , Luke Jeagal  4 , Nazi Torabi  5 , Dick Menzies  1   4   6 , Alexei Korobitsyn  7 , Madhukar Pai  1   4   6 , Ruvandhi R Nathavitharana  8 , Faiz Ahmad Khan  1   4   6
Affiliations

A systematic review of the diagnostic accuracy of artificial intelligence-based computer programs to analyze chest x-rays for pulmonary tuberculosis

Miriam Harris et al. PLoS One. .

Abstract

We undertook a systematic review of the diagnostic accuracy of artificial intelligence-based software for identification of radiologic abnormalities (computer-aided detection, or CAD) compatible with pulmonary tuberculosis on chest x-rays (CXRs). We searched four databases for articles published between January 2005-February 2019. We summarized data on CAD type, study design, and diagnostic accuracy. We assessed risk of bias with QUADAS-2. We included 53 of the 4712 articles reviewed: 40 focused on CAD design methods ("Development" studies) and 13 focused on evaluation of CAD ("Clinical" studies). Meta-analyses were not performed due to methodological differences. Development studies were more likely to use CXR databases with greater potential for bias as compared to Clinical studies. Areas under the receiver operating characteristic curve (median AUC [IQR]) were significantly higher: in Development studies AUC: 0.88 [0.82-0.90]) versus Clinical studies (0.75 [0.66-0.87]; p-value 0.004); and with deep-learning (0.91 [0.88-0.99]) versus machine-learning (0.82 [0.75-0.89]; p = 0.001). We conclude that CAD programs are promising, but the majority of work thus far has been on development rather than clinical evaluation. We provide concrete suggestions on what study design elements should be improved.

PubMed Disclaimer

Conflict of interest statement

The authors have declared that no competing interests exist.

Figures

Fig 1
Fig 1. Study flow diagram.
Computer aided detection (CAD).
Fig 2
Fig 2. Quality assessment (QUADAS 2) graph of development studies.
Fig 3
Fig 3. Quality assessment (QUADAS 2) graph of clinical studies.
Fig 4
Fig 4. Forest plots of accuracy measures of development and CAD4TB studies.
TP, true positive; FP, false positive; FN, false negative; TN, true negative; AI, artificial intelligence; CXRs, chest x-rays; ML, machine learning; DL, deep learning; CI, confidence interval; NAAT, nucleic acid amplification test.
Fig 5
Fig 5. Boxplots of the AUC of studies stratified by software design, CXR usage, reference standard, and degree of patient selection, index test, and reference standard bias.
AUC, area under the cure; Vs, versus; CXR, chest x-ray.
See this image and copyright information in PMC

References

    1. WHO. Global Tuberculosis Report 2017. Geneva: Licence: CC BY-NC- SA 3.0 IGO.: World Health Organization, 2017.
    1. WILLIAMS FH. The Use of X-Ray Examinations in Pulmonary Tuberculosis. The Boston Medical and Surgical Journal. 1907;157(26):850–3. 10.1056/nejm190712261572602 - DOI
    1. Pande T, Pai M, Khan FA, Denkinger CM. Use of chest radiography in the 22 highest tuberculosis burden countries. Eur Respir J. 2015;46(6):1816–9. 10.1183/13993003.01064-2015 - DOI - PubMed
    1. Chunhaswasdikul B, Kamolratanakul P, Jittinandana A, Tangcharoensathien V, Kuptawintu S, Pantumabamrung P. Anti-tuberculosis programs in Thailand: a cost analysis. Southeast Asian J Trop Med Public Health. 1992;23(2):195–9. Epub 1992/06/01. . - PubMed
    1. WHO. Chest Radiography in Tuberculosis Detection—Summary of current WHO recommendations and guidance on programmatic approaches. 2016 2016. Report No.

Publication types

MeSH terms