Deep learning radiomics for focal liver lesions diagnosis on long-range contrast-enhanced ultrasound and clinical factors

Abstract

Background: Routine clinical factors play an important role in the clinical diagnosis of focal liver lesions (FLLs); however, they are rarely used in computer-assisted diagnosis. Therefore, we developed a deep learning (DL) radiomics model, and investigated its effectiveness in diagnosing FLLs using long-range contrast-enhanced ultrasound (CEUS) cines and clinical factors.

Methods: Herein, 303 patients with pathologically confirmed FLLs after surgery at three hospitals were retrospectively enrolled and divided into a training cohort (n=203), internal validation (IV) cohort (n=50) from one hospital with the ratio of 4:1, and external validation (EV) cohort (n=50) from the other two hospitals. Four DL radiomics models, namely Four Stream 3D convolutional neural network (FS3D^U) (trained with CEUS cines only), FS3D^U+A (trained with CEUS cines and alpha fetoprotein), FS3D^U+H (trained with CEUS cines and hepatitis), and FS3D^U+A+H (trained with CEUS cines, alpha fetoprotein, and hepatitis), were formed based on 3D convolutional neural networks (CNNs). They used approximately 20-s preoperative CEUS cines and/or clinical factors to extract spatiotemporal features for the classification of FLLs and the location of the region of interest. The area under curve of the receiver operating characteristic and diagnosis speed were calculated to evaluate the models in the IV and EV cohorts, and they were compared with those of two radiologists. Two-sided Delong tests were used to calculate the statistical differences between the models and radiologists.

Results: FS3D^U+A+H, which incorporated CEUS cines, hepatitis, and alpha fetoprotein, achieved the highest area under curve of 0.969 (95% CI: 0.901-1.000) and 0.957 (95% CI: 0.894-1.000) among radiologists and other models in IV and EV cohorts, respectively. A significant difference was observed when comparing FS3D^U and radiologist 2 (all P<0.05). The diagnosis speed of all the models was the same (10.76 s per patient), and it was two times faster than those of the radiologists (radiologist 1: 23.74 and 27.75 s; radiologist 2: 25.95 and 29.50 s in IV and EV cohorts, respectively).

Conclusions: The proposed DL radiomics demonstrated excellent performance on the benign and malignant diagnosis of FLLs by combining CEUS cines and clinical factors. It could help the individualized characterization of FLLs, and enhance the accuracy of diagnosis in the future.

Keywords: Deep learning (DL); contrast-enhanced ultrasound (CEUS); diagnosis; focal liver lesions (FLLs); radiomics.

Figure 1

Flowchart of the study from data collection to evaluation. CEUS, contrast-enhanced ultrasound; IV, internal validation; EV, external validation; FS3D, Four-Stream three-dimensional.

Figure 2

Pre-processing of CEUS cines. CEUS, contrast-enhanced ultrasound.

Figure 3

Four-stream three-dimensional convolutional neural network composed of four steps: input, feature extraction, feature selection, and output. CEUS cines and images were normalized as input and predicted values of malignant lesions were calculated as output. I3D, inflated three dimentional; CSN, channel-separated convolutional networks; CNN, convolutional neural network; CEUS, contrast-enhanced ultrasound; AFP, alpha fetoprotein.

Figure 4

ROC curves of the final model in the (A) training dataset, (B) IV dataset, and (C) EV dataset. The red five-pointed star indicates the AUC values of a radiologist with 12 years of experience. The red dot indicates the AUC values of a radiologist with 5 years of experience. ROC, receiver operating characteristic curve; IV, internal validation; EV, external validation; AUC, area under the receiver operating characteristic curve.

Figure 5

Feature visualization. There are two samples: the first and third rows show continuous frames of CEUS on a malignant lesion and a benign lesion, respectively. The lesions on the CEUS are marked by a white asterisk. The second and fourth rows show the corresponding feature maps of the lesions, marked by a white arrow. Sample [1], obtained from a 28-year-old man with liver cirrhosis, exhibited a malignant lesion of dimensions 38 mm × 35 mm (HCC) in the right liver, with an AFP concentration of 4.22 ng/mL. The imaging features showed rapid hyper-enhancement from the periphery to the center of the lesion in the artery phase and iso-enhancement in the portal venous phase. Sample [2], obtained from a 32-year-old woman with no history of hepatitis, exhibited a benign lesion of dimensions 41 mm × 27 mm (FNH) in the right liver, with an AFP concentration of 2.5 ng/mL. The imaging features showed slow hyper-enhancement from the center to the periphery in the late artery phase and wash-out in the late portal venous phase. CEUS, contrast-enhanced ultrasound; HCC, hepatocellular carcinoma; FNH, focal nodular hyperplasia.

Figure 6

The misclassified cases reported by the model. Case A of hemangioma was misdiagnosed as a malignant tumor. Case B of cholangiocarcinoma was misdiagnosed as a benign tumor. Case C of primary liver cancer was misdiagnosed as a benign tumor. The lesions on the CEUS are marked by a white asterisk. CEUS, contrast-enhanced ultrasound.