Hybrid-FHR: a multi-modal AI approach for automated fetal acidosis diagnosis

Abstract

Background: In clinical medicine, fetal heart rate (FHR) monitoring using cardiotocography (CTG) is one of the most commonly used methods for assessing fetal acidosis. However, as the visual interpretation of CTG depends on the subjective judgment of the clinician, this has led to high inter-observer and intra-observer variability, making it necessary to introduce automated diagnostic techniques.

Methods: In this study, we propose a computer-aided diagnostic algorithm (Hybrid-FHR) for fetal acidosis to assist physicians in making objective decisions and taking timely interventions. Hybrid-FHR uses multi-modal features, including one-dimensional FHR signals and three types of expert features designed based on prior knowledge (morphological time domain, frequency domain, and nonlinear). To extract the spatiotemporal feature representation of one-dimensional FHR signals, we designed a multi-scale squeeze and excitation temporal convolutional network (SE-TCN) backbone model based on dilated causal convolution, which can effectively capture the long-term dependence of FHR signals by expanding the receptive field of each layer's convolution kernel while maintaining a relatively small parameter size. In addition, we proposed a cross-modal feature fusion (CMFF) method that uses multi-head attention mechanisms to explore the relationships between different modalities, obtaining more informative feature representations and improving diagnostic accuracy.

Results: Our ablation experiments show that the Hybrid-FHR outperforms traditional previous methods, with average accuracy, specificity, sensitivity, precision, and F1 score of 96.8, 97.5, 96, 97.5, and 96.7%, respectively.

Conclusions: Our algorithm enables automated CTG analysis, assisting healthcare professionals in the early identification of fetal acidosis and the prompt implementation of interventions.

Keywords: Attention mechanisms; Cardiotocography; Cross-modal feature fusion; Fetal acidosis; Fetal heart rate; Multi-modal; Temporal convolutional network.

Fig. 1

Overview of proposed method (Hybrid-FHR)

Fig. 2

A Dilated Causal Convolution with dilation factors d = 1, 2, 3 and kernel size k = 3

Fig. 3

The internal structure of SE-TCNBlock

Fig. 4

The internal structure of CMFF

Fig. 5

The distribution of the neonatal umbilical artery pH

Fig. 6

Comparison of original signal (Top) and denoised (Bottom) signal. Outliers and missing values are removed from FHR signals using a mini-batch-based minimized sparse dictionary learning approach

Fig. 7

Effect of different kernel sizes (left) and num_heads (right) on the model

Fig. 8

The comparison results of the Accuracy of different Signal Backbone models on the validation set

Fig. 9

The boxplots of the accuracy of different Signal Backbone models on the test set. The numbers in brackets on the x-axis indicate the total number of parameters for each signal backbone model. SD stands for standard deviation

Fig. 10

The Visualization Output of each Layer in the Hybrid-FHR using t-SNE