ECG Paper Record Digitization and Diagnosis Using Deep Learning

Abstract

Purpose: Electrocardiogram (ECG) is one of the most essential tools for detecting heart problems. Till today most of the ECG records are available in paper form. It can be challenging and time-consuming to manually assess the ECG paper records. Hence, automated diagnosis and analysis are possible if we digitize such paper ECG records.

Methods: The proposed work aims to convert ECG paper records into a 1-D signal and generate an accurate diagnosis of heart-related problems using deep learning. Camera-captured ECG images or scanned ECG paper records are used for the proposed work. Effective pre-processing techniques are used for the removal of shadow from the images. A deep learning model is used to get a threshold value that separates ECG signal from its background and after applying various image processing techniques threshold ECG image gets converted into digital ECG. These digitized 1-D ECG signals are then passed to another deep learning model for the automated diagnosis of heart diseases into different classes such as ST-segment elevation myocardial infarction (STEMI), Left Bundle Branch Block (LBBB), Right Bundle Branch Block (RBBB), and T-wave abnormality.

Results: The accuracy of deep learning-based binarization is 97%. Further deep learning-based diagnosis approach of such digitized paper ECG records was having an accuracy of 94.4%.

Conclusions: The digitized ECG signals can be useful to various research organizations because the trends in heart problems can be determined and diagnosed from preserved paper ECG records. This approach can be easily implemented in areas where such expertise is not available.

Supplementary information: The online version contains supplementary material available at 10.1007/s40846-021-00632-0.

Keywords: Deep learning; Diagnosis; Digitization; Paper ECG.

Fig. 1

Flow diagram for automated binarization and diagnosis using deep learning from 12-lead paper ECG records. The 12 lead was converted to a single lead, pre-processed for shadow removal, and the Level of Binarization (LOB) characteristic curve is extracted. Then the DL model is used, and the background is removed. The signal is post-processed to remove the labels. The 1-D signal is then extracted using vertical scanning. A deep learning-based diagnosis was done at the end

Fig. 2

Concept of LOB characteristic curve a–c thresholding of sample grayscale image for a particular level of Binarization (LOB) from 0 to 255, d–h normalized sum (NS) varies from 0 to 1 for complete black to complete white image, i–n binarized images at different threshold values (LOB) and NS, o Chessboard image (as shown in the snippet) and its corresponding characteristics curve showing the step transition from black to white, p image with different Grayscale levels and its corresponding characteristics curve, q characteristics curve of single-lead ECG (shown in the snippet) showing two slopes, one for ECG signal and other for background grid

Fig. 3

a Deep learning Model for determining threshold value. The input is LOB characteristic curve of size is $1\times 255$. The 1-D signal is passed through multiple Dense layers. The output is then passed through a dropout layer. The data is then passed through the Dense model of dimension $1\times 1$ and we get the predicted output threshold value. b Deep learning Model for diagnosis. The input signal of the size $400\times 1\times 1$ is passed to the convolution layer along with the ReLU layer of dimension $3\times 16$. After that, it is passed through a series of 384, and two fully connected layers along with hidden layers. Then it is passed through the Softmax layer which further diagnoses the ECG report

Fig. 4

1-D ECG signal extraction from an input image of ECG paper record irrespective of the background (textbfa), (f), (k) ECG signal recorded on different background color grids. b, g, l binarized images after thresholding. Images after dilation and skeletonizing (c), (h), (m) Images after character removal (d), (i), (n). Final 1-D signal extraction using vertical scanning (e), (j), (o)

Fig. 5

a The camera captured paper ECG record with shadow. Results after binarization (b), skeletonization and character removal (c) and 1-D signal extraction (d) from images with shadow. f–h extraction of 1-D signal from the image with non-uniform illumination shown in (e). Binarized ECG image with undesired lead names (shown in i and k, red box), removal of lead names j, l that avoids distortion in signal extraction

Fig. 6

The confusion matrix for the deep learning model is used for diagnosis of the heart abnormalities