Representation learning for clinical time series prediction tasks in electronic health records

Abstract

Background: Electronic health records (EHRs) provide possibilities to improve patient care and facilitate clinical research. However, there are many challenges faced by the applications of EHRs, such as temporality, high dimensionality, sparseness, noise, random error and systematic bias. In particular, temporal information is difficult to effectively use by traditional machine learning methods while the sequential information of EHRs is very useful.

Method: In this paper, we propose a general-purpose patient representation learning approach to summarize sequential EHRs. Specifically, a recurrent neural network based denoising autoencoder (RNN-DAE) is employed to encode inhospital records of each patient into a low dimensional dense vector.

Results: Based on EHR data collected from Shuguang Hospital affiliated to Shanghai University of Traditional Chinese Medicine, we experimentally evaluate our proposed RNN-DAE method on both mortality prediction task and comorbidity prediction task. Extensive experimental results show that our proposed RNN-DAE method outperforms existing methods. In addition, we apply the "Deep Feature" represented by our proposed RNN-DAE method to track similar patients with t-SNE, which also achieves some interesting observations.

Conclusion: We propose an effective unsupervised RNN-DAE method to summarize patient sequential information in EHR data. Our proposed RNN-DAE method is useful on both mortality prediction task and comorbidity prediction task.

Keywords: Electronic health records; Mortality prediction; Recurrent neural network; Representation learning.

Fig. 1

An overview of the proposed representation learning approach to generate patient vectors and further applications

Fig. 2

Three different forms of the representation of patients. Here, patient may have various inpatient times (e.g., x,y,z). The tensor representation of each patient consists of multiple multi-hot vectors of N-dimensions (i.e., N=1309). The statistic-based representation is derived by operating summary statistics, and it gets a vector with N-dimensions. Typically, distributed representation is a better representation with D-dimensions (i.e., D=300), where D is much lower than N. a Tensor representation of patients. b Statistic-based representation of patients. c Distributed representation of patients

Fig. 3

The architecture of our proposed RNNDAE model. Multi-hot vectors (x_t) with time series are added by a Gaussian noise and then encoded by a GRU_encoder model into the patient vector (c). Given the patient vector, another GRU_decoder model is used to decode in order to make the input (x_t) and the output (y_t) are consistent as much as possible

Fig. 4

A diagram of t-SNE technique for dimensionality reduction. With the help of t-SNE, some D-dimensional data points are projected into 2-dimensional space. Specially, the red points indicate the patients who finally die and the blue ones represent those patients who do not die

Fig. 5

Results of patient similarity analysis based on “Deep Feature”

Fig. 6

Comparative results of different sampling strategies

Fig. 7

Comparative results of different binary classifiers

Fig. 8

Comparative results of patient representation vectors with different dimensions

Fig. 9

The effect of different training data sizes