Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2021 Dec 13;11(1):23895.
doi: 10.1038/s41598-021-03265-0.

Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis

Andrew J Codlin  1 , Thang Phuoc Dao  2 , Luan Nguyen Quang Vo  3   2 , Rachel J Forse  3   4 , Vinh Van Truong  5 , Ha Minh Dang  5 , Lan Huu Nguyen  5 , Hoa Binh Nguyen  6 , Nhung Viet Nguyen  6 , Kristi Sidney-Annerstedt  4 , Bertie Squire  7 , Knut Lönnroth  4 , Maxine Caws  7   8
Affiliations
Comparative Study

Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis

Andrew J Codlin et al. Sci Rep. .

Abstract

There have been few independent evaluations of computer-aided detection (CAD) software for tuberculosis (TB) screening, despite the rapidly expanding array of available CAD solutions. We developed a test library of chest X-ray (CXR) images which was blindly re-read by two TB clinicians with different levels of experience and then processed by 12 CAD software solutions. Using Xpert MTB/RIF results as the reference standard, we compared the performance characteristics of each CAD software against both an Expert and Intermediate Reader, using cut-off thresholds which were selected to match the sensitivity of each human reader. Six CAD systems performed on par with the Expert Reader (Qure.ai, DeepTek, Delft Imaging, JF Healthcare, OXIPIT, and Lunit) and one additional software (Infervision) performed on par with the Intermediate Reader only. Qure.ai, Delft Imaging and Lunit were the only software to perform significantly better than the Intermediate Reader. The majority of these CAD software showed significantly lower performance among participants with a past history of TB. The radiography equipment used to capture the CXR image was also shown to affect performance for some CAD software. TB program implementers now have a wide selection of quality CAD software solutions to utilize in their CXR screening initiatives.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Figure 1
Figure 1
ROC graphs for each CAD software. ROC AUC area under the receiver operating characteristic curve.
Figure 2
Figure 2
Diagram of test library creation.
See this image and copyright information in PMC

References

    1. World Health Organization . Global Tuberculosis Report 2020. World Health Organization; 2020.
    1. World Health Organization . The End TB Strategy. World Health Organization; 2015.
    1. Creswell J, Sahu S, Blok L, Bakker MI, Stevens R, Ditiu L. A multi-site evaluation of innovative approaches to increase tuberculosis case notification: Summary results. PLoS ONE. 2014;9(4):e94465. - PMC - PubMed
    1. Miller C, Lonnroth K, Sotgiu G, Migliori GB. The long and winding road of chest radiography for tuberculosis detection. Eur. Respir. J. 2017;49(5):1700364. - PubMed
    1. World Health Organization . WHO Consolidated Guidelines on Tuberculosis Module 2: Screening—Systematic Screening for Tuberculosis Disease. World Health Organization; 2021. - PubMed

Publication types