Diagnostic accuracy of convolutional neural networks in classifying hepatic steatosis from B-mode ultrasound images: a systematic review with meta-analysis and novel validation in a community setting in Telangana, India

Abstract

Background: Ultrasound is a widely available, inexpensive, and non-invasive modality for evaluating hepatic steatosis (HS). However, the scarcity of radiological expertise limits its utility. Convolutional Neural Networks (CNNs) have potential for automated classification of HS using B-mode ultrasound images. We aimed to assess their diagnostic accuracy and generalisability across diverse study settings and populations.

Methods: We systematically reviewed two biomedical databases up to Dec 12, 2023, to identify studies that applied CNNs in the classification of HS using B-mode ultrasound images as input (PROSPERO: CRD42024501483). We supplemented this review with a novel analysis of the community-based Andhra Pradesh Children and Parents' Study (APCAPS) in India to address the overrepresentation of hospital samples and lack of data on South Asian populations who exhibit a distinct central adiposity phenotype that could influence CNN performance. We quantitatively synthesised diagnostic accuracy metrics for eligible studies using random-effects meta-analyses.

Findings: Our search returned 289 studies, of which 17 were eligible. All but one of the 17 studies were based in hospital or clinical outpatient settings with curated cases and controls. Studies were conducted exclusively in East Asian, European, or North American populations. Studies employed varying gold standards: seven studies (41.18%) used liver biopsy, three (17.64%) used MRI proton density fat fraction, and seven (41.18%) used clinician-evaluated ultrasound-based HS grades. The APCAPS sample included 219 participants with radiologist-assigned HS grades. Across the range of study settings and populations, CNNs demonstrated good diagnostic accuracy. Meta-analysis of studies with low risk of bias reporting on five unique datasets showed a pooled area under the receiver operating characteristic curve of 0.93 (95% CI 0.73-0.98) for detecting any severity and 0.86 (95% CI 0.77-0.92) for detecting moderate-to-severe HS severity grades, respectively.

Interpretation: CNNs have good diagnostic accuracy and generalisability for HS classification, suggesting potential for real-world application.

Funding: Medical Research Council, UK (MR/T038292/1, MR/V001221/1).

Keywords: Artificial intelligence; Convolutional neural networks; Fatty liver disease; Machine learning; Ultrasonography.

Fig. 1

Andhra Pradesh Children and Parents' Study (APCAPS) liver ultrasound gold standard dataset generation. The binary label refers to (S2/S3) vs (S0/S1). Reported N for images refers to numbers before data augmentation. ∗As of August 2022.

Fig. 2

Study selection. Numbers are accurate as of Dec 12, 2023. US: Ultrasound, MRI-PDFF: Magnetic Resonance Imaging Proton Density Fat Fraction.

Fig. 3

Forest plot of study datasets (n = 5) reporting AUC with 95% CIs for any severity HS detection task (S1 or higher) vs S0 HS, included in the meta-analysis. AUC: Area under the receiver operating characteristic curve, HS: Hepatic Steatosis.

Fig. 4

Class Activation Maps (CAM) for a randomly selected True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN) prediction. The target class predicted is moderate-to-severe (S2/S3) HS. Red areas represent image regions that are most relevant for model prediction, with higher intensities (darker red) corresponding to higher importance. Similarly, blue areas represent regions that the model considers least relevant for prediction, with higher intensities corresponding to lower importance. Plots reveal that the model focuses on mid- and far-fields (the bottom 2/3) of the ultrasound image, primarily on the bulk of the hepatic parenchyma including the diaphragm and portal vasculature.

Fig. 5

Forest plot of study datasets (n = 5) AUC with 95% CIs for moderate-to-severe HS detection task, (S2/S3) vs (S0/S1), included in the meta-analysis. AUC: Area under the receiver operating characteristic curve, CI: Confidence Interval, HS: Hepatic Steatosis.