Machine learning-based combined nomogram for predicting the risk of pulmonary invasive fungal infection in severely immunocompromised patients

Abstract

Background: Early and accurate diagnosis of invasive fungal infection (IFI) is pivotal for the initiation of effective antifungal therapy for patients with hematologic malignancies.

Methods: This retrospective study involved 235 patients with hematologic malignancies and pulmonary infections diagnosed as IFIs (n=118) or bacterial pneumonia (n=117). Patients were randomly divided into training (n=188) and validation (n=47) datasets. Four feature selection methods with nine classifiers were implemented to select the optimal machine learning (ML) model using five-fold cross-validation. A radiomic signature was constructed using a linear ML algorithm, and a radiomic score (Radscore) was calculated. The combined model was developed with the Radscore, the significant clinical and radiologic factors were selected using multivariable logistic regression, and the results were presented as a clinical radiomic nomogram. A prospective pilot study was also conducted to compare the classification performance of the combined nomogram with practicing radiologists.

Results: Significant differences were found in the Radscore between IFI and bacterial pneumonia patients in the training (0.683 vs. -0.724, P<0.001) and validation set (0.353 vs. -0.717, P=0.002). The combined model showed good discrimination performance in the validation cohort [area under the curve (AUC) =0.844] and outperformed the clinical (AUC =0.696) and radiomics (AUC =0.767) model alone (both P<0.05).

Conclusions: The clinical radiomic nomogram can serve as a promising predictive tool for IFI in patients with hematologic malignancies.

Keywords: Machine learning (ML); computed tomography (CT); hematologic malignancy; invasive fungal infection (IFI); radiomics.

Figure 1

The flowchart of patient recruitment. CT, computed tomography; IFI, invasive fungal infection.

Figure 2

The workflow of data analysis. ROC, receiver operator characteristic; AUC, area under the ROC curve; GGO, ground-glass opacity; IFI, invasive fungal infection; DCA, decision curve analysis; CT, computed tomography.

Figure 3

Heatmaps of the AUCs from different combinations of feature selection methods (columns) and classification algorithms (rows) for 2D and 3D radiomics features in the 5-fold cross-validation (A) and validation (B) dataset. ANOVA, analysis of variance; KW, Kruskal-Wallis rank-sum test; RFE, recursive feature elimination; NB, naïve Bayes; AE, Adaboost ensemble; GP, Gaussian process; DT, decision tree; LR, logistic regression; SVM, support vector machine; RF, random forest; LDA, linear discriminant analysis; LASSO, least absolute shrinkage and selection operator; AUC, area under the receiver operator characteristic curve.

Figure 4

ROC curves of the clinical, radiomics, and combined models in the training (A) and validation (B) dataset. AUC, area under the receiver operator characteristic curve; ROC, receiver operator characteristic.

Figure 5

Nomogram developed with the clinical radiomic model and calibration curves. (A) The developed clinical radiomic nomogram for predicting the probability of IFIs. (B,C) Calibration curves for predicting IFIs in the training and validation cohorts. Calibration curves indicate the goodness of fit of the constructed nomogram. The predictive performance of the nomogram (red line) closer to the ideal prediction line (45° gray line) represents a higher predictive accuracy of the nomogram. GGO, ground-glass opacity; IFI, invasive fungal infection.

Figure 6

The decision curve analysis for the clinical, radiomic, and combined models in the training and validation cohorts. The y-axis indicates the net benefit; the x-axis indicates threshold probability. The gray line represents the decision curve of the assumption that all patients have IFIs (“treat all”); the black line represents the decision curve of the assumption that no patients have an IFI (“treat none”). The farther the decision curve is from the 2 extreme lines, the higher the clinical decision net benefit of the model. IFI, invasive fungal infection.

Figure 7

Comparison of diagnostic performance between the combined nomogram and practicing radiologists in a prospective fashion. IFI, invasive fungal infection.

Figure 8

Representative cases to show diagnostic ability of the combined model and practicing radiologists for IFIs. (A) A 69-year-old woman with acute myelogenous leukemia and IFI. The mass was correctly diagnosed as an IFI by the combined model and practicing radiologists. (B) A 23-year-old man with acute lymphatic leukemia and IFI. The lesion was correctly diagnosed as an IFI by the combined model but was misdiagnosed by the 2 radiologists. (C) A 67-year-old man with acute myelogenous leukemia and bacterial pneumonia. The lesion was correctly diagnosed by the combined model and the senior radiologist but was misdiagnosed by the junior radiologist. IFI, invasive fungal infection; +, positive; −, negative.