Integrating real-world data and machine learning: A framework to assess covariate importance in real-world use of alternative intravenous dosing regimens for atezolizumab

Abstract

The increase in the availability of real-world data (RWD), in combination with advances in machine learning (ML) methods, provides a unique opportunity for the integration of the two to explore complex clinical pharmacology questions. Here we present a recently developed RWD/ML framework that utilizes ML algorithms to understand the influence and importance of various covariates on the use of a given dose and schedule for drugs that have multiple approved dosing regimens. To demonstrate the application of this framework, we present atezolizumab as a use case on account of its three approved alternative intravenous (IV) dosing regimens. As expected, the real-world use of atezolizumab has generally been increasing since 2016 for the 1200 mg every 3 weeks regimen and since 2019 for the 1680 mg every 4 weeks regimen. Out of the ML algorithms evaluated, XGBoost performed the best, as measured by the area under the precision-recall curve, with an emphasis on the under-sampled class given the imbalance in the data. The importance of features was measured by Shapley Additive exPlanations (SHAP) values and showed metastatic breast cancer and use of protein-bound paclitaxel as the most correlated with the use of 840 mg every 2 weeks. Although patient usage data for alternative IV dosing regimens are still maturing, these analyses provide initial insights on the use of atezolizumab and set up a framework for the re-analysis of atezolizumab (at a future data cut) as well as application to other molecules with approved alternative dosing regimens.

FIGURE 1

Machine learning workflow to assess alternative dosing patterns using RWD. Selected cohort is split into training data (used for model development) and test data (used for model assessment). Based on predicted class distribution, the model is evaluated for performance. Shapley Additive exPlanations (SHAP) values are calculated to quantify feature contribution to RWD‐based outcomes. RWD, real‐world data; ML, machine learning; SES, socioeconomic status. Created in BioRender. Velasquez, E. (2024) https://biorender.com/o63s907.

FIGURE 2

Overview of data cleaning and cohort selection workflow. Compliance score is calculated as (number of intervals at most commonly used dosing regimen)/(total number of intervals). Data cutoff date: February 2024.

FIGURE 3

Use of different atezolizumab intravenous dosing regimens over time shown (a) without 1200 mg Q3W and (b) with 1200 mg Q3W. Data from 2024 not shown in plots given early data cutoff (February 2024).

FIGURE 4

Precision–Recall (PR) curves for three binary classifiers. 840 mg Q2W is the positive class, and 1200 mg Q3W is the negative class. Models are randomized into training and test datasets with a 70/30 ratio, respectively. PR curves were generated using the test dataset and only PR curves for single bootstrap are shown. XGBoost has the highest AP of the three models and is used for the rest of our assessments, with a mean [90% confidence interval] AP Score of 0.9956 [0.9936, 0.9971] as compared to 0.9192 [0.9069, 0.9298] for Random Forest and 0.9228 [0.9059, 0.9381] for CatBoost. RF, Random Forest; XG, XGBoost; Cat, CatBoost; AP, average precision.

FIGURE 5

SHAP value summaries for XGBoost model. (a) Beeswarm plot of features ranked highest to lowest from average magnitude of importance. Example SHAP values (portrayed using waterfall plots) for one patient belonging to the (b) negative class (1200 mg Q3W) and another patient belonging to the (c) positive class (840 mg Q2W). Waterfall plots summarize SHAP contributions, starting from the average prediction (E[f(x)], at the bottom of the plot) and ending at the individual prediction (f(x), at the top of the plot); features are listed as most to least important (top to bottom) and include their respective magnitude/directionality. f(x) closer to 0 represents an individual prediction of 840 mg Q2W, while f(x) closer to 1 represents an individual prediction of 1200 mg Q3W. Only the top 20 features are shown in (a).