Automatic Real-Time Detection and Diagnosis of Liver Tumor with Ultrasound

Abstract

Background/aim: Ultrasonography is the most commonly used screening tool for hepatocellular carcinoma (HCC). However, the diagnostic performance of ultrasound is highly operator dependent. We aimed to develop deep learning (DL) models to automatically diagnose and detect hepatic lesions in a larger dataset, with HCC as the dominant malignancy.

Methods: We enrolled patients diagnosed with hepatic tumors using abdominal ultrasound between January 2002 and December 2020 in a retrospective cohort with a diagnosis of malignant and benign lesions. A total of 1576 patients with 4599 images and 6001 lesions were analyzed. Deep learning models included ResNet50, Xception, Inception Resnet V2, EfficientNet-B5, EfficientNetV2-S, EfficientNetV2-L, Swin-T, and Swin-B for diagnosis and YOLOR for lesion detection. We analyzed the area under the curve (AUC) to determine the diagnostic performance and choose the best model. The mean Average Precision (mAP) score was then evaluated for real-time lesion detection using the area under the precision-recall curve after the average of each category.

Results: The dataset was separated into 1061 in training, 373 in validation, and 142 testing sets. The AUC for ResNet50, Xception, Inception Resnet V2, EfficientNet-B5, EfficientNetV2-S, EfficientNetV2-L, Swin-T, and Swin-B are 0.88, 0.89, 0.88, 0.90, 0.85, 0.89, 0.89, and 0.90, respectively. The mAP scores for detecting and differentiating malignant and benign lesions for YOLOR-W6 and YOLOR-D6 in the validation and testing sets were 0.5134/0.5342 and 0.5410/0.5631.

Conclusion: Our study demonstrated that DL models can differentiate between benign and malignant lesions with high accuracy on ultrasound images. Simultaneous DL-based lesion detection and classification are also possible using real-time ultrasonography.

Keywords: artificial intelligence; deep learning; diagnosis; liver neoplasms; ultrasonography.

Figure 1

Diagnostic performance for four deep learning algorithms, including (A) ResNet50, (B) Xception, (C) InceptionResnetV2, and (D) EfficientNet-B5.

Figure 2

Diagnostic performance in validation and testing for EfficientNetV2-S (A and B), EfficientNetV2-L (C and D), Swin-T (E and F), and Swin-B (G and H).

Figure 3

The loss to differentiate between malignant and benign lesions in training and validation for YOLOR-W6 (A) and YOLOR-D6 (B). The mean Average Precision score to differentiate between malignant and benign lesions and classify 8 kinds of hepatic lesions in validation and testing for YOLOR-W6 (C) and YOLOR-D6 (D).

Figure 4

Visual presentation ultrasound images with predictive bounding box. The blue/red bounding boxes represented benign (A–C) and malignant (D–F) lesions. The lesions can be detected at left lobe (A), split view (B), right lobe (C), two lesions in one view (D) and sine images (F).