Development of an artificial intelligence-based multimodal model for assisting in the diagnosis of necrotizing enterocolitis in newborns: a retrospective study

Abstract

Objective: The purpose of this study is to develop a multimodal model based on artificial intelligence to assist clinical doctors in the early diagnosis of necrotizing enterocolitis in newborns.

Methods: This study is a retrospective study that collected the initial laboratory test results and abdominal x-ray image data of newborns (non-NEC, NEC) admitted to our hospital from January 2022 to January 2024.A multimodal model was developed to differentiate multimodal data, trained on the training dataset, and evaluated on the validation dataset. The interpretability was enhanced by incorporating the Gradient-weighted Class Activation Mapping (GradCAM) analysis to analyze the attention mechanism of the multimodal model, and finally compared and evaluated with clinical doctors on external datasets.

Results: The dataset constructed in this study included 11,016 laboratory examination data from 408 children and 408 image data. When applied to the validation dataset, the area under the curve was 0.91, and the accuracy was 0.94. The GradCAM analysis shows that the model's attention is focused on the fixed dilatation of the intestinal folds, intestinal wall edema, interintestinal gas, and portal venous gas. External validation demonstrated that the multimodal model had comparable accuracy to pediatric doctors with ten years of clinical experience in identification.

Conclusion: The multimodal model we developed can assist doctors in early and accurate diagnosis of NEC, providing a new approach for assisting diagnosis in underdeveloped medical areas.

Keywords: artificial intelligence; attention analysis; computer-aided diagnosis; multimodal model; necrotizing enterocolitis.

Figure 1

Study design flowchart.

Figure 2

Performance evaluation of ResNet34-clinical model. (A) Confusion matrix of ResNet34-Clinical on the validation dataset. (B) Receiver operating characteristic curves of the ResNet34-Clinical model on the validation dataset. (C) Receiver operating characteristic curves of the ResNet34-Clinical model on the external independent dataset. (D) Receiver operating characteristic curves of clinical doctors on the external independent dataset.

Figure 3

ResNet34-Clinical grad-CAM images (A) Overlaid images of abdominal x-rays and GradCAM for NEC patients. (B) Display of overlaid images of abdominal x-rays and GradCAM for non-NEC newborns. The thicker red area indicates the image portions that ResNet34-Clinical focuses on during the classification of NEC patients and healthy non-NEC newborns.