Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Sep;6(9):e605-e613.
doi: 10.1016/S2589-7500(24)00118-3. Epub 2024 Jul 19.

Computer-aided detection of tuberculosis from chest radiographs in a tuberculosis prevalence survey in South Africa: external validation and modelled impacts of commercially available artificial intelligence software

Zhi Zhen Qin  1 , Martie Van der Walt  2 , Sizulu Moyo  3 , Farzana Ismail  4 , Phaleng Maribe  3 , Claudia M Denkinger  5 , Sarah Zaidi  6 , Rachael Barrett  6 , Lindiwe Mvusi  7 , Nkateko Mkhondo  8 , Khangelani Zuma  3 , Samuel Manda  2 , Lisa Koeppel  5 , Thuli Mthiyane  2 , Jacob Creswell  6
Affiliations

Computer-aided detection of tuberculosis from chest radiographs in a tuberculosis prevalence survey in South Africa: external validation and modelled impacts of commercially available artificial intelligence software

Zhi Zhen Qin et al. Lancet Digit Health. 2024 Sep.

Erratum in

Abstract

Background: Computer-aided detection (CAD) can help identify people with active tuberculosis left undetected. However, few studies have compared the performance of commercially available CAD products for screening in high tuberculosis and high HIV settings, and there is poor understanding of threshold selection across products in different populations. We aimed to compare CAD products' performance, with further analyses on subgroup performance and threshold selection.

Methods: We evaluated 12 CAD products on a case-control sample of participants from a South African tuberculosis prevalence survey. Only those with microbiological test results were eligible. The primary outcome was comparing products' accuracy using the area under the receiver operating characteristic curve (AUC) against microbiological evidence. Threshold analyses were performed based on pre-defined criteria and across all thresholds. We conducted subgroup analyses including age, gender, HIV status, previous tuberculosis history, symptoms presence, and current smoking status.

Findings: Of the 774 people included, 516 were bacteriologically negative and 258 were bacteriologically positive. Diverse accuracy was noted: Lunit and Nexus had AUCs near 0·9, followed by qXR, JF CXR-2, InferRead, Xvision, and ChestEye (AUCs 0·8-0·9). XrayAME, RADIFY, and TiSepX-TB had AUC under 0·8. Thresholds varied notably across these products and different versions of the same products. Certain products (Lunit, Nexus, JF CXR-2, and qXR) maintained high sensitivity (>90%) across a wide threshold range while reducing the number of individuals requiring confirmatory diagnostic testing. All products generally performed worst in older individuals, people with previous tuberculosis, and people with HIV. Variations in thresholds, sensitivity, and specificity existed across groups and settings.

Interpretation: Several previously unevaluated products performed similarly to those evaluated by WHO. Thresholds differed across products and demographic subgroups. The rapid emergence of products and versions necessitates a global strategy to validate new versions and software to support CAD product and threshold selections.

Funding: Government of Canada.

PubMed Disclaimer

Conflict of interest statement

Declaration of interests CMD declares research grants from US National Institutes of Health, German Ministry of Education and Research, German Alliance for Global Health Research, United States Agency for International Development, FIND, German Center for Infection Research, and WHO. CMD also declares that she serves as an academic editor for PLoS Medicine and on the WHO Technical Advisory Group on tuberculosis diagnostics. All other authors declare no competing interests.

Figures

Figure 1
Figure 1
Overview of the data collection from the first national tuberculosis prevalence survey, South Africa Bac+=bacteriologically positive. Bac-=bacteriologically negative. *Bac+ and Bac- denote the bacteriological status determined by the composite microbiological reference standard in this study, where anyone with Mycobacterium tuberculosis detected by either Xpert Ultra or culture is referred to as Bac+.
Figure 2
Figure 2
The receiver operating characteristic curves against the composite microbiology reference standard (A) and the AUCs (B) of 12 commercially available CAD products AUC=area under the receiver operating characteristic curve. CAD=computer-aided detection.
See this image and copyright information in PMC

References

    1. WHO . World Health Organization; Geneva: 2023. Global tuberculosis report 2023.
    1. van Cleeff MRA, Kivihya-Ndugga LE, Meme H, et al. The role and performance of chest x-ray for the diagnosis of tuberculosis: a cost-effective analysis in Nairobi, Kenya. BMC Infect Dis. 2005;5:111. - PMC - PubMed
    1. van Cleeff MRA, Kivihya-Ndugga L, Githui W, Nganga L, Odhiambo J, Klatser PR. A comprehensive study of the efficiency of the routine pulmonary tuberculosis diagnostic process in Nairobi. Int J Tuberc Lung Dis. 2003;7:186–189. - PubMed
    1. Pinto LM, Pai M, Dheda K, Schwartzman K, Menzies D, Steingart KR. Scoring systems using chest radiographic features for the diagnosis of pulmonary tuberculosis in adults: a systematic review. Eur Respir J. 2013;42:480–494. - PubMed
    1. Graham S, Das Gupta K, Hidvegi R J, et al. Chest radiograph abnormalities associated with tuberculosis: reproducibility and yield of active cases. Int J Tuberc Lung Dis. 2002;6:137–142. - PubMed

Publication types

MeSH terms

LinkOut - more resources