Diagnostic Accuracy of Chest X-ray Computer-Aided Detection Software for Detection of Prevalent and Incident Tuberculosis in Household Contacts

Abstract

Background: World Health Organization (WHO) tuberculosis (TB) screening guidelines recommend computer-aided detection (CAD) software for chest radiograph (CXR) interpretation. However, studies evaluating their diagnostic and prognostic accuracy are limited.

Methods: We conducted a prospective cohort study of household contacts of rifampicin-resistant TB in South Africa. Participants underwent baseline CXR and sputum investigation (routine [single spontaneous] and enhanced [additionally 2-3 induced]) for prevalent TB and follow-up for incident TB. Three CXR-CAD software products (CAD4TBv7.0, qXRv3.0.0, and Lunit INSIGHT v3.1.4.111) were compared. We evaluated their performance to detect routine and enhanced prevalent and incident TB, comparing performance with blood tests (Xpert MTB host-response, erythrocyte sedimentation rate, C-reactive protein, QuantiFERON) in a subgroup.

Results: 483 participants were followed up for 4.6 years (median). There were 23 prevalent (7 routinely diagnosed) and 38 incident TB cases. The AUC ROCs (95% CIs) to identify prevalent TB for CAD4TBv7.0, qXRv3.0.0, and Lunit INSIGHT v3.1.4.111 were .87 (.77-.96), .88 (.79-.97), and .91 (.83-.99), respectively. More than 30% with scores above recommended CAD thresholds who were bacteriologically negative on routine baseline sputum were subsequently diagnosed by enhanced sputum investigation or during follow-up. The AUC performance of baseline CAD to identify incident cases ranged between .60 and .65. Diagnostic performance of CAD for prevalent TB was superior to blood testing.

Conclusions: Our findings suggest that the potential of CAD-CXR screening for TB is not maximized as a high proportion of those above current thresholds, but with a negative routine confirmatory sputum, have true TB disease that may benefit intervention.

Keywords: active case finding; asymptomatic; chest X-ray; computer-aided detection; tuberculosis.

Figure 1.

Study population. Details of participants who were included in this analysis including a description of TB disease status. Abbreviations: CAD, computer-aided detection; CXR, chest X-ray; PHDC, Provincial Health Data Centre; TB, tuberculosis.

Figure 2.

Yields of TB-screening strategies. In this representation of the yields of these screenings, each dot represents 1 study participant. The color coding of the participants represents TB disease status as defined by bacteriological investigation. Participants have also been grouped by HIV status to show the relative representation of TB in each population. The figure is accompanied by a table, which shows the sensitivity and specificity of the different screening approaches (the numbers represent proportions and 95% CIs). Abbreviations: CAD4TB, CAD4TB version 7; CXR, chest-X-ray; HIV, human immunodeficiency virus; Lunit INSIGHT CXR, Lunit INSIGHT CXR version 3.1.4.111; qXR, qXR version 3.0.0; TB, tuberculosis.

Figure 3.

Performance of 3 different CAD software products for the detection of prevalent and incident TB cases. The CAD software products evaluated were CAD4TB version 7.0 (CAD4TB; Delft Imaging, ‘s-Hertogenbosch, Netherlands), qXR version 3.0.0 (qXR; Qure.ai, Mumbai, India), and Lunit INSIGHT CXR version 3.1.4.111 (Lunit; Lunit, Seoul, South Korea). Routine prevalent TB was defined as those initiated on TB treatment in whom Mycobacterium tuberculosis (Mtb) was detected using Xpert MTB/RIF on the first single, spontaneously produced baseline sputum sample; all prevalent TB was defined as those initiated on TB treatment in whom Mtb was detected using Xpert MTB/RIF and/or culture on any other baseline sputum sample; and incident TB was defined as those cases initiated on TB treatment in whom Mtb was detected in at least 1 follow-up sputum sample by Xpert MTB/RIF and/or culture or where the initiation of TB treatment was based on clinical grounds. Accuracy was measured using the AUC ROC curve. The numbers represent proportions (95% CI). Abbreviations: AUC ROC, area under the receiver operating characteristic; CAD, computer-aided detection; CAD4TB, CAD4TB version 7; Lunit INSIGHT CXR, Lunit INSIGHT CXR version 3.1.4.111; qXR, qXR version 3.0.0; TB, tuberculosis.

Figure 4.

Representation of study participants above and below the manufacturer’s or commonly used threshold for each CAD software (CAD4TB version 7 [CAD4TB], qXR version 3.0.0 [qXR], and Lunit INSIGHT CXR version 3.1.4.111 [Lunit]) by TB status. The figure provides a visual representation of the distribution of CAD software scores for each software used (CAD4TB, qXR, Lunit), broken down by TB disease status. Panels under (A) include all 483 participants and panels under (B) include the 374 participants without a previous history of TB. Routine prevalent TB was defined as those initiated on TB treatment in whom Mycobacterium tuberculosis (Mtb) was detected using Xpert MTB/RIF on the first single, spontaneously produced baseline sputum sample; enhanced prevalent TB was defined as those initiated on TB treatment in whom Mtb was detected using Xpert MTB/RIF and/or culture on any other baseline sputum sample not included as routine prevalent; and incident TB was defined as those cases initiated on TB treatment in whom Mtb was detected in at least 1 follow-up sputum sample by Xpert MTB/RIF and/or culture or where the initiation of TB treatment was based on clinical grounds. Each dot represents a study participant, and the color coding of the dot represents the bacteriological status of that participant. For example, some incident cases are represented by black dots, indicating that these participants were bacteriologically negative; these represent the cases of incident TB that were diagnosed clinically. Horizontal lines have been added to represent 3 thresholds—the threshold above which the score is consistent with TB infection as recommended by the manufacturer or commonly used in practice, the threshold derived by using the WHO TPP optimal specificity for a TB triage test, and the threshold derived by using the WHO TPP optimal sensitivity for a TB triage test. The latter 2 thresholds were generated using all study participants/all prevalent cases of TB. Abbreviations: CAD, computer-aided detection; Lunit INSIGHT CXR, Lunit INSIGHT CXR version 3.1.4.111; qXR, qXR version 3.0.0; TB, tuberculosis; TPP, target product profile; WHO TPP, world health organization target product profile.

Figure 5.

Representation of participants with baseline CAD scores for pulmonary TB above the manufacturer’s recommended thresholds (or in the case of CAD4TBv7, a commonly used threshold). Each participant with a score above recommended thresholds at which a diagnosis of TB is likely is represented as a human figure. Each figure is accurately represented as male or female, the proportions of participants with prevalent (routine and enhanced) and incident TB are shown and those with symptoms at baseline or a history of previous TB highlighted. Numbers represent proportions (95% CI). The chest radiograph illustrations show an example of the typical output from each of the CAD products. Abbreviations: CAD, computer-aided detection; CAD4TBv7/CAD4TB, CAD4TB version 7; F, female; Lunit v3/Lunit INSIGHT CXR, Lunit INSIGHT CXR version 3.1.4.111; M, male; qXR v3/qXR, qXR version 3.0.0; TB, tuberculosis.