A mobile-optimized artificial intelligence system for gestational age and fetal malpresentation assessment

Abstract

Background: Fetal ultrasound is an important component of antenatal care, but shortage of adequately trained healthcare workers has limited its adoption in low-to-middle-income countries. This study investigated the use of artificial intelligence for fetal ultrasound in under-resourced settings.

Methods: Blind sweep ultrasounds, consisting of six freehand ultrasound sweeps, were collected by sonographers in the USA and Zambia, and novice operators in Zambia. We developed artificial intelligence (AI) models that used blind sweeps to predict gestational age (GA) and fetal malpresentation. AI GA estimates and standard fetal biometry estimates were compared to a previously established ground truth, and evaluated for difference in absolute error. Fetal malpresentation (non-cephalic vs cephalic) was compared to sonographer assessment. On-device AI model run-times were benchmarked on Android mobile phones.

Results: Here we show that GA estimation accuracy of the AI model is non-inferior to standard fetal biometry estimates (error difference -1.4 ± 4.5 days, 95% CI -1.8, -0.9, n = 406). Non-inferiority is maintained when blind sweeps are acquired by novice operators performing only two of six sweep motion types. Fetal malpresentation AUC-ROC is 0.977 (95% CI, 0.949, 1.00, n = 613), sonographers and novices have similar AUC-ROC. Software run-times on mobile phones for both diagnostic models are less than 3 s after completion of a sweep.

Conclusions: The gestational age model is non-inferior to the clinical standard and the fetal malpresentation model has high AUC-ROCs across operators and devices. Our AI models are able to run on-device, without internet connectivity, and provide feedback scores to assist in upleveling the capabilities of lightly trained ultrasound operators in low resource settings.

Keywords: Health care; Medical research.

Fig. 1. Development of an artificial intelligence system to acquire and interpret blind-sweep ultrasound for antenatal diagnostics.

a Datasets were curated from sites in Zambia and the USA and include ultrasound acquired by sonographers and midwives. Ground truth for gestational age was derived from the initial exam as part of clinical practice. An artificial intelligence (AI) system was trained to identify gestational age and fetal malpresentation and was evaluated by comparing the accuracy of AI predictions with the accuracy of clinical standard procedures. The AI system was developed using only sonographer blind-sweep data, and its generalization to novice users was tested on midwife data. Design of the AI system considered suitability for deployment in low-to-middle-income countries in three ways: first, the system interpreted ultrasound from low-cost portable ultrasound devices; second, near real-time interpretation is available offline on mobile phone devices; and finally, the AI system produces feedback scores that can be used to provide feedback to users. b Blind-sweep ultrasound acquisition procedure. The procedure can be performed by novices with a few hours of ultrasound training. While the complete protocol involves six sweeps, a set of two sweeps (M and R) were found to be sufficient for maintaining the accuracy of gestational age estimation.

Fig. 2. Gestational age estimation.

n = 407 study participants, blind sweeps performed by expert sonographers. a Blind-sweep procedure and standard fetal biometry procedure absolute error versus ground truth gestational age (4-week windows). Box indicates 25th, 50th, and 75th percentile absolute error, and whiskers indicate 5th and 95th percentile absolute error. b Error distributions for blind-sweep procedure and standard fetal biometry procedure. c Paired errors for a blind sweep and standard fetal biometry estimates in the same study visit. The errors of the two methods exhibit correlation, but the worst-case errors for the blind-sweep procedure have a lower magnitude than the standard fetal biometry method. d Video sequence feedback-score calibration on the test sets. The realized model estimation error on held-out video sequences decreases as the model’s feedback score increases. A thresholded feedback score may be used as a user feedback signal to redo low-quality blind sweeps. Box indicates 25th, 50th, and 75th percentile of absolute errors, and whiskers indicate the 5th and 95th percentile absolute error.

Fig. 3. Fetal malpresentation estimation.

n = 623 study participants. Receiver operating characteristic (ROC) curves for fetal malpresentation estimation. Crosses indicate the predefined operating point selected from the tuning dataset. a ROC comparison based on the type of device: low-cost and standard. b ROC comparison based on the type of ultrasound operator: novices and sonographers.