Deep learning on time series laboratory test results from electronic health records for early detection of pancreatic cancer

Abstract

The multi-modal and unstructured nature of observational data in Electronic Health Records (EHR) is currently a significant obstacle for the application of machine learning towards risk stratification. In this study, we develop a deep learning framework for incorporating longitudinal clinical data from EHR to infer risk for pancreatic cancer (PC). This framework includes a novel training protocol, which enforces an emphasis on early detection by applying an independent Poisson-random mask on proximal-time measurements for each variable. Data fusion for irregular multivariate time-series features is enabled by a "grouped" neural network (GrpNN) architecture, which uses representation learning to generate a dimensionally reduced vector for each measurement set before making a final prediction. These models were evaluated using EHR data from Columbia University Irving Medical Center-New York Presbyterian Hospital. Our framework demonstrated better performance on early detection (AUROC 0.671, CI 95% 0.667 - 0.675, p < 0.001) at 12 months prior to diagnosis compared to a logistic regression, xgboost, and a feedforward neural network baseline. We demonstrate that our masking strategy results greater improvements at distal times prior to diagnosis, and that our GrpNN model improves generalizability by reducing overfitting relative to the feedforward baseline. The results were consistent across reported race. Our proposed algorithm is potentially generalizable to other diseases including but not limited to cancer where early detection can improve survival.

Keywords: Early detection of cancer; Electronic Health Records; Machine learning; Pancreatic cancer.

Fig. 1.

Data preprocessing flow chart. We obtained 458,252 patient samples with 30,195 lab variables from CUIMC-NYP EHR data using our data retrieval criteria (Table 1) and processed them into the final dataset composed of patient cohort where PC patients who received pancreatitis diagnosis before PC and nonPC patients with pancreatitis were eliminated (Total 9,057 patients where 834 are PC patients).

Fig. 2.

The evaluation of data structure discrepancies between PC and nonPC. (A) Among the selected PC (1,200) and nonPC (161,849), we investigated data structure in terms of data completeness and average number of measurements for each variable and their discrepancies between PC and nonPC, which can introduce bias in the prediction model. We sorted the 418 lab variables by data completeness of PC group and the panel A shows data structure of top 50 lab variables. (B) We selected the first 33 (panel A gray dotted line) lab variables, where the measurement system changes from Cerner system to NYP. The discrepancies in data completeness between PC and nonPC decreased when we filtered patients with the patient group who have at least one of the selected 33 lab variables. We then assigned random diagnosis dates for nonPC patients and configured the nonPC dataset into pre-diagnosis based on the average percentage reduction calculated from the process of configuring PC dataset into pre-diagnosis data. This led the average number of measurements of nonPC group close to that of PC group as shown in panel B.

Fig. 3.

A grouped deep neural network (GrpNN) incorporated with a random masking strategy. (A) GrpNN, where time series measurements of thirty three lab variables individually pass through the embedding network, producing a new representation of each variable in 4-dimensions, which is then put together (i.e., merge) to pass through the prediction network for making a prediction. (B) The process of the random masking strategy applied to each variable, incorporated during GrpNN training. The random samples are generated every training epoch and applied to the batch data before fed into GrpNN model. (C) The resultant masked histograms for the exemplary lab variables (‘Creatinine’ and ‘Magnesium level’) show reduced discrepancies in the number of lab measurements between PC and nonPC.

Fig. 4.

The propensity score matching. The baseline characteristics are potential confounders as they can be reflected in lab measurements. In the dataset composed of 126,655 nonPC and 835 PC (Fig. 1) before the propensity matching procedure, the separability resulting from the baseline characteristics (i.e., race, ethnicity, sex, zip code, patient language, age, smoking, obesity, diabetes) were 72.9%. By applying propensity score matching, we reduced separability of our final dataset to 54.6%.

Fig. 5.

Early detection performance. We evaluated early detection performance of GrpNN model with and without random masking strategy (GrpNN vs GrpNN+RM). (A) A flowchart for describing dataset preparation for estimating prediction score at x months prior to diagnosis date. (B) The early detection performance significantly improves with the random masking strategy. (C) The post hoc analysis of GrpNN+RM on the hold-out set stratified by race. The early detection performance was consistent across reported race. The abbreviations for the race, [W, A, B, O, U], represent [White, Asian, Black, Other Combinations not described, Unknown].