Automated MR Image Prescription of the Liver Using Deep Learning: Development, Evaluation, and Prospective Implementation

Abstract

Background: There is an unmet need for fully automated image prescription of the liver to enable efficient, reproducible MRI.

Purpose: To develop and evaluate artificial intelligence (AI)-based liver image prescription.

Study type: Prospective.

Population: A total of 570 female/469 male patients (age: 56 ± 17 years) with 72%/8%/20% assigned randomly for training/validation/testing; two female/four male healthy volunteers (age: 31 ± 6 years).

Field strength/sequence: 1.5 T, 3.0 T; spin echo, gradient echo, bSSFP.

Assessment: A total of 1039 three-plane localizer acquisitions (26,929 slices) from consecutive clinical liver MRI examinations were retrieved retrospectively and annotated by six radiologists. The localizer images and manual annotations were used to train an object-detection convolutional neural network (YOLOv3) to detect multiple object classes (liver, torso, and arms) across localizer image orientations and to output corresponding 2D bounding boxes. Whole-liver image prescription in standard orientations was obtained based on these bounding boxes. 2D detection performance was evaluated on test datasets by calculating intersection over union (IoU) between manual and automated labeling. 3D prescription accuracy was calculated by measuring the boundary mismatch in each dimension and percentage of manual volume covered by AI prescription. The automated prescription was implemented on a 3 T MR system and evaluated prospectively on healthy volunteers.

Statistical tests: Paired t-tests (threshold = 0.05) were conducted to evaluate significance of performance difference between trained networks.

Results: In 208 testing datasets, the proposed method with full network had excellent agreement with manual annotations, with median IoU > 0.91 (interquartile range < 0.09) across all seven classes. The automated 3D prescription was accurate, with shifts <2.3 cm in superior/inferior dimension for 3D axial prescription for 99.5% of test datasets, comparable to radiologists' interreader reproducibility. The full network had significantly superior performance than the tiny network for 3D axial prescription in patients. Automated prescription performed well across single-shot fast spin-echo, gradient-echo, and balanced steady-state free-precession sequences in the prospective study.

Data conclusion: AI-based automated liver image prescription demonstrated promising performance across the patients, pathologies, and field strengths studied.

Evidence level: 4.

Technical efficacy: Stage 1.

Keywords: AI; automated scan prescription; deep learning; image prescription; liver.

Figure 1.

Flow-chart summary for data retrieval, annotation, training, prescription, evaluation, and scanner implementation. Manual labeling consisted of 7 localization regions (LiverAx, TorsoAx, ArmsAx, LiverCor, TorsoCor, LiverSag, TorsoSag) that was evaluated by assessment of inter-reader reproducibility. A convolutional neural network (CNN) for object detection was trained with 80% of the datasets. The minimum 3D box needed to cover the appropriate labeled 2D boxes was used to obtain the final 3D automated prescription. Evaluation of 2D and 3D boxes was performed on the remaining 20% datasets across patients, pathologies, and acquisition settings. We successfully implemented the method on a clinical MR system and conducted a prospective study with 6 healthy volunteers across acquisition sequences. LiverAx/TorsoAx/ArmsAx: liver/torso/arms in the axial view; LiverCor/TorsoCor: liver/torso in the coronal view; LiverSag/TorsoSag: liver/torso in the sagittal view.

Figure 2.

3D detection of the liver in all three localizer orientations, with examples of common pathologies. a) In most cases, the liver volume was covered accurately by automated prescription. The automated prescription aligned well with the manual annotation in patients with focal lesions (b), iron overload (c), ascites (d), and/or cirrhosis (e), as well as in a patient with splenomegaly (f), where the spleen abuts the liver and its signal level is similar to that of liver, but the proposed method was still able to identify the liver correctly.

Figure 3.

Examples of algorithm failures for 3D liver detection. a) Missed coverage in the tip of the lateral segment of the left lobe due to insufficient axial localizer slices. Inaccurate automated object detection was observed for patients with multiple renal cysts (b) or liver cysts (c), or with dielectric shading (b,c) with signal dropouts in the central portion of multiple localizer images. Variation in manual annotation between two radiologists and even between repeated efforts by the same radiologist was observed in severe cases such as (c).

Figure 4.

Accuracy of 2D annotation across all label classes (a), 3D liver detection (b), and 3D image prescription (c-e). IoU histograms for all classes are qualitatively similar, with IoU median >0.91 and interquartile range <0.09. In (b-e), x axis shows the 6 edges: right (R), left (L), posterior (P), anterior (A), inferior (I), and superior (S); y axis shows the difference between automated and manual volumes (0: perfect alignment; negative offset: AI covering more volume; positive offset: missed volume). All boxes are tight around 0. For 3D axial prescription, the shift in the S/I dimension was less than 2.3 cm for 99.5% of the test datasets.

Figure 5.

As training size increased, the percentage of test cases with high overlap (>90%) in 3D between AI and manual prescription increased for 3D liver detection and axial prescription. AI performance for 3D axial prescription plateaued after training with 500 patients’ datasets (60% of training data). AI performance for 3D liver detection approached but never reached radiologists’ inter-reader reproducibility performance. Training with at least 250 datasets (30% of training data), AI-based 3D axial prescription performed better than (manual) inter-reader reproducibility.

Figure 6.

The AI-based automated image prescription was successfully implemented on a clinical MRI system for prospective scanning. This online implementation performed well across various localizer sequences for 6 healthy volunteers. Automated prescription demonstrated promising performance for 3D axial prescription (similar to that of the retrospective study) across SSFSE, GRE, and bSSFP sequences with various parameters. The performance of AI-based automated prescription was best when using spin-echo (SSFSE) acquisitions.