RadioLOGIC, a healthcare model for processing electronic health records and decision-making in breast disease

Abstract

Digital health data used in diagnostics, patient care, and oncology research continue to accumulate exponentially. Most medical information, and particularly radiology results, are stored in free-text format, and the potential of these data remains untapped. In this study, a radiological repomics-driven model incorporating medical token cognition (RadioLOGIC) is proposed to extract repomics (report omics) features from unstructured electronic health records and to assess human health and predict pathological outcome via transfer learning. The average accuracy and F1-weighted score for the extraction of repomics features using RadioLOGIC are 0.934 and 0.934, respectively, and 0.906 and 0.903 for the prediction of breast imaging-reporting and data system scores. The areas under the receiver operating characteristic curve for the prediction of pathological outcome without and with transfer learning are 0.912 and 0.945, respectively. RadioLOGIC outperforms cohort models in the capability to extract features and also reveals promise for checking clinical diagnoses directly from electronic health records.

Keywords: artificial intelligence; breast cancer; decision support; digital health data; electronic health records; radiology; repomics.

Graphical abstract

Figure 1

Characteristics of all radiological report content for the training cohort and the independent test cohort in this study (A) Training cohort. (B) Independent test cohort. A “0” means not applicable or missing. For margin: 1 for irregular, 2 for circumscribed. For shape: 1 for irregular, 2 for oval/round. Density category A (1): the breasts are almost entirely fatty. Density category B (2): there are scattered areas of fibroglandular density. Density category C (3): heterogeneously dense. Density category D (4): extremely dense. See also Table S1.

Figure 2

Visualizations of words and sentence (A) Word cloud based on all radiological reports. (B) Visualization of word co-occurrence. (C) Association of top 50 co-occurrence words. (D) Associations between words in a given report after pre-training. (E) The correlation between the selected word and other words in the report. See also Figures S1–S3 and Table S2.

Figure 3

Metrics details for extracting repomics features in the independent test cohort (A) Accuracy. (B) Micro F1 score. (C) Macro F1 score. (D) Weighted F1 score. The black vertical lines in the graph represent 95% confidence intervals.

Figure 4

Examples of repomics feature extraction from corresponding images and radiological reports (A) Mammography. (B) Ultrasound. (C) MRI. “∗∗∗” in the radiological report indicates the patient’s private information. See also Figures S4–S7.

Figure 5

Prediction results for downstream tasks (A) Confusion matrix results for predicting BI-RADS scores in the independent test cohort using RadioLOGIC without transfer learning. (B) Confusion matrix results for predicting BI-RADS scores in the independent test cohort using RadioLOGIC via transfer learning. (C) Receiver operating characteristic curves for predicting pathological outcome in the independent test cohort using RNN. (D) Receiver operating characteristic curves for predicting pathological outcome in the independent test cohort using RadioLOGIC. The 95% confidence intervals are shown as a shaded area for the ROC curve. ATT, attention mechanism; BI-RADS, breast imaging-reporting and data system; RadioLOGIC, radiological repomics-driven model incorporating medical token cognition; RNN, recurrent neural networks; TF, transfer learning. See also Table S3.