MI-DenseCFNet: deep learning-based multimodal diagnosis models for Aureus and Aspergillus pneumonia

Abstract

Objective: To build and merge a diagnostic model called multi-input DenseNet fused with clinical features (MI-DenseCFNet) for discriminating between Staphylococcus aureus pneumonia (SAP) and Aspergillus pneumonia (ASP) and to evaluate the significant correlation of each clinical feature in determining these two types of pneumonia using a random forest dichotomous diagnosis model. This will enhance diagnostic accuracy and efficiency in distinguishing between SAP and ASP.

Methods: In this study, 60 patients with clinically confirmed SAP and ASP, who were admitted to four large tertiary hospitals in Kunming, China, were included. Thoracic high-resolution CT lung windows of all patients were extracted from the picture archiving and communication system, and the corresponding clinical data of each patient were collected.

Results: The MI-DenseCFNet diagnosis model demonstrates an internal validation set with an area under the curve (AUC) of 0.92. Its external validation set demonstrates an AUC of 0.83. The model requires only 10.24s to generate a categorical diagnosis and produce results from 20 cases of data. Compared with high-, mid-, and low-ranking radiologists, the model achieves accuracies of 78% vs. 75% vs. 60% vs. 40%. Eleven significant clinical features were screened by the random forest dichotomous diagnosis model.

Conclusion: The MI-DenseCFNet multimodal diagnosis model can effectively diagnose SAP and ASP, and its diagnostic performance significantly exceeds that of junior radiologists. The 11 important clinical features were screened in the constructed random forest dichotomous diagnostic model, providing a reference for clinicians.

Clinical relevance statement: MI-DenseCFNet could provide diagnostic assistance for primary hospitals that do not have advanced radiologists, enabling patients with suspected infections like Staphylococcus aureus pneumonia or Aspergillus pneumonia to receive a quicker diagnosis and cut down on the abuse of antibiotics.

Key points: • MI-DenseCFNet combines deep learning neural networks with crucial clinical features to discern between Staphylococcus aureus pneumonia and Aspergillus pneumonia. • The comprehensive group had an area under the curve of 0.92, surpassing the proficiency of junior radiologists. • This model can enhance a primary radiologist's diagnostic capacity.

Keywords: Artificial intelligence; Communicable diseases; Deep learning; Pneumonia.

Fig. 1

^AThe first affiliated hospital of kunming medical university, the first people’s hospital of yunnan province, and the kunming yan’an hospital; ^BKunming first people’s hospital ganmei hospital

Fig. 2

(A) Preprocessed CT image set; (B) Skeleton network DenseNet-201 for extracting feature vectors from CT image sets; (C) Input of clinical information (clinical symptoms, laboratory results, imaging features) is fed into a three-layer deep neural network (DNN) for training and extracting clinical feature vectors; (D) Converging and connecting clinical and image feature vectors; (E) After fusing clinical and image feature vector information, it is fed into a DNN with two layers for training, and finally, a DNN with two neurons outputs the classification and diagnosis results; (F) Confusion matrix visualization classification diagnosis results

Fig. 3

Image-based: Image-only group model [DenseNet-201]; Clinical Joint: Integrated group (clinical+imaging) model [MI-DenseCFNet]

Fig. 4

A–D and a–d are the CT images of suspicious lesion areas detected by the deep learning diagnostic model for ASP and SAP, respectively. The color bar on the right panel indicates the intensity of attention, with a darker red color indicating the strongest attention and a darker blue color indicating weaker attention

Fig. 5

External validation set ROC curve

Fig. 6

A–C Model diagnostic performance cases of ASP and SAP; a–c confusion matrix