Clinical Inflection Point Detection on the Basis of EHR Data to Identify Clinical Trial-Ready Patients With Cancer

Abstract

Purpose: To inform precision oncology, methods are needed to use electronic health records (EHRs) to identify patients with cancer who are experiencing clinical inflection points, consistent with worsening prognosis or a high propensity to change treatment, at specific time points. Such patients might benefit from real-time screening for clinical trials.

Methods: Using serial unstructured imaging reports for patients with solid tumors or lymphoma participating in a single-institution precision medicine study, we trained a deep neural network natural language processing (NLP) model to dynamically predict patients' prognoses and propensity to start new palliative-intent systemic therapy within 30 days. Model performance was evaluated using Harrell's c-index (for prognosis) and the area under the receiver operating characteristic curve (AUC; for new treatment and new clinical trial enrollment). Associations between model outputs and manual annotations of cancer progression were also evaluated using the AUC.

Results: A deep NLP model was trained and evaluated using 302,688 imaging reports for 16,780 patients. In a held-out test set of 34,770 reports for 1,952 additional patients, the model predicted survival with a c-index of 0.76 and initiation of new treatment with an AUC of 0.77. Model-generated prognostic scores were associated with annotation of cancer progression on the basis of manual EHR review (n = 1,488 reports for 110 patients with lung or colorectal cancer) with an AUC of 0.78, and predictions of new treatment were associated with annotation of cancer progression on the basis of manual EHR review with an AUC of 0.84.

Conclusion: Training a deep NLP model to identify clinical inflection points among patients with cancer is feasible. This approach could identify patients who may benefit from real-time targeted clinical trial screening interventions at health system scale.

FIG 1.

Distribution of predicted log hazard of mortality by actual subsequent 6-month mortality (n = 34,770 imaging reports for 1,952 patients). Histogram of model output 1 (predicted log hazard of mortality per year), by actual 6-month survival, following each imaging report, demonstrating that patients with worse predicted prognoses at any given time point have a lower rate of 6-month survival following that time point. Test set, restricted to imaging reports at least 6 months before the cohort censoring date of December 31, 2018.

FIG 2.

Distribution of predicted log odds of new treatment in 30 days versus actual subsequent treatment in 30 days (n = 34,770 imaging reports for 1,952 patients). Histogram of model output 2 (predicted log odds of new treatment within 30 days), by actual rate of new treatment within 30 days, following each imaging report, demonstrating that patients with higher predicted odds of initiating new treatment after any given time point had a higher actual rate of initiating new treatment. Test set, restricted to imaging reports at least 6 months before the cohort censoring date of December 31, 2018.

FIG 3.

Distribution of predicted log hazards of mortality versus manually annotated cancer progression among patients with non–small-cell lung cancer or colorectal cancer (n = 1,488 imaging reports for 110 patients). Histogram of model output 1 (predicted log hazards of mortality per year), by manual annotation of the presence or absence of cancer progression on each imaging report, demonstrating that patients with higher predicted log hazards of mortality at any given time point were more likely to have cancer progression manually annotated at that time point. Test set, restricted to imaging reports at least 6 months before the cohort censoring date of December 31, 2018.

FIG 4.

Distribution of predicted log odds of new treatment within 30 days versus manually annotated cancer progression among patients with non–small-cell lung cancer or colorectal cancer (n = 1,488 imaging reports for 110 patients). Histogram of model output 2 (predicted log odds of new palliative-intent systemic therapy within 30 days), by manual annotation of the presence or absence of cancer progression on each imaging report, demonstrating that patients with a higher predicted log odds of new treatment at any given time point were more likely to have cancer progression manually annotated at that time point. Test set, restricted to imaging reports at least 6 months before the cohort censoring date of December 31, 2018.