Machine-Learning Parsimonious Prediction Model for Diagnostic Screening of Severe Hematological Adverse Events in Cancer Patients Treated with PD-1/PD-L1 Inhibitors: Retrospective Observational Study by Using the Common Data Model

Abstract

Background/Objectives: Earlier detection of severe immune-related hematological adverse events (irHAEs) in cancer patients treated with a PD-1 or PD-L1 inhibitor is critical to improving treatment outcomes. The study aimed to develop a simple machine learning (ML) model for predicting irHAEs associated with PD-1/PD-L1 inhibitors. Methods: We utilized the Observational Medical Outcomes Partnership-Common Data Model based on electronic medical records from a tertiary (KHMC) and a secondary (KHNMC) hospital in South Korea. Severe irHAEs were defined as Grades 3-5 by the Common Terminology Criteria for Adverse Events (version 5.0). The predictive model was developed using the KHMC dataset, and then cross-validated against an independent cohort (KHNMC). The full ML models were then simplified by selecting critical features based on the feature importance values (FIVs). Results: Overall, 397 and 255 patients were included in the primary (KHMC) and cross-validation (KHNMC) cohort, respectively. Among the tested ML algorithms, random forest achieved the highest accuracy (area under the receiver operating characteristic curve [AUROC] 0.88 for both cohorts). Parsimonious models reduced to 50% FIVs of the full models showed comparable performance to the full models (AUROC 0.83-0.86, p > 0.05). The KHMC and KHNMC parsimonious models shared common predictive features including furosemide, oxygen gas, piperacillin/tazobactam, and acetylcysteine. Conclusions: Considering the simplicity and adequate predictive performance, our simplified ML models might be easily implemented in clinical practice with broad applicability. Our model might enhance early diagnostic screening of irHAEs induced by PD-1/PD-L1 inhibitors, contributing to minimizing the risk of severe irHAEs and improving the effectiveness of cancer immunotherapy.

Keywords: PD-1 inhibitor; PD-L1 inhibitor; common data model (CDM); immune checkpoint inhibitor; immune-related hematological adverse events; machine learning; observational medical outcomes partnership (OMOP); parsimonious model; pharmacovigilance; real-world data (RWD); risk prediction.

Figure 1

Flowchart of the study population selection: (a) Primary model development cohort selection (i.e., KHMC). (b) Cross-comparison cohort selection (i.e., KHNMC). Abbreviations: irHAE, immune-related hematological adverse event; KHMC, Kyung Hee Medical Center; KHNMC, Gangdong Kyung Hee University Medical Center; OMOP-CDM, Observational Medical Outcomes Partnership–Common Data Model; PD-1 inhibitor, programmed cell death 1 inhibitor; PD-L1 inhibitor, programmed cell death ligand 1 inhibitor.

Figure 1

Flowchart of the study population selection: (a) Primary model development cohort selection (i.e., KHMC). (b) Cross-comparison cohort selection (i.e., KHNMC). Abbreviations: irHAE, immune-related hematological adverse event; KHMC, Kyung Hee Medical Center; KHNMC, Gangdong Kyung Hee University Medical Center; OMOP-CDM, Observational Medical Outcomes Partnership–Common Data Model; PD-1 inhibitor, programmed cell death 1 inhibitor; PD-L1 inhibitor, programmed cell death ligand 1 inhibitor.

Figure 2

Comparative predictive performance of the parsimonious prediction models based on random forest as well as the full prediction model for (a) the primary model development cohort (i.e., KHMC) and (b) the cross-comparison cohort (i.e., KHNMC). Abbreviations: 100% Model, the full prediction model; 75% Model, the parsimonious model with reduced features with the sum of their importance values to be 75% of the full model; 60% Model, the parsimonious model with reduced features with the sum of their importance values to be 60% of the full model; 50% Model, the parsimonious model with reduced features with the sum of their importance values to be 50% of the full model; AUROC, area under the receiver operating characteristic curve; p-value from the DeLong test for comparison to the 100% full model based on each cohort; KHMC: Kyung Hee Medical Center; KHNMC: Gangdong Kyung Hee University Medical Center.

Figure 2

Comparative predictive performance of the parsimonious prediction models based on random forest as well as the full prediction model for (a) the primary model development cohort (i.e., KHMC) and (b) the cross-comparison cohort (i.e., KHNMC). Abbreviations: 100% Model, the full prediction model; 75% Model, the parsimonious model with reduced features with the sum of their importance values to be 75% of the full model; 60% Model, the parsimonious model with reduced features with the sum of their importance values to be 60% of the full model; 50% Model, the parsimonious model with reduced features with the sum of their importance values to be 50% of the full model; AUROC, area under the receiver operating characteristic curve; p-value from the DeLong test for comparison to the 100% full model based on each cohort; KHMC: Kyung Hee Medical Center; KHNMC: Gangdong Kyung Hee University Medical Center.

Figure 3

Importance of features included in the final parsimonious model for (a) the primary model development cohort (i.e., KHMC) and (b) the cross-comparison cohort (i.e., KHNMC). DCSI: diabetes complications severity index; FIV: feature importance value; MABP: mean arterial blood pressure.

Figure 3

Importance of features included in the final parsimonious model for (a) the primary model development cohort (i.e., KHMC) and (b) the cross-comparison cohort (i.e., KHNMC). DCSI: diabetes complications severity index; FIV: feature importance value; MABP: mean arterial blood pressure.