Exploring the discrepancies between clinical trials and real-world data: A small-cell lung cancer study

Abstract

The potential of real-world data to inform clinical trial design and supplement control arms has gained much interest in recent years. The most common approach relies on reproducing control arm outcomes by matching real-world patient cohorts to clinical trial baseline populations. However, recent studies pointed out that there is a lack of replicability, generalisability, and consensus. In this article, we propose a novel approach that aims to explore and examine these discrepancies by concomitantly investigating the impact of selection criteria and operations on the measurements of outcomes from the patient data. We tested the approach on a dataset consisting of small-cell lung cancer patients receiving platinum-based chemotherapy regimens from a real-world data cohort (n = 223) and six clinical trial control arms (n = 1224). The results showed that the discrepancy between real-world and clinical trial data potentially depends on differences in both patient populations and operational conditions (e.g., frequency of assessments, and censoring), for which further investigation is required. Discovering and accounting for confounders, including hidden effects of differences in operations related to the treatment process and clinical trial study protocol, would potentially allow for improved translation between clinical trials and real-world data. Continued development of the method presented here to systematically explore and account for these differences could pave the way for transferring learning across clinical studies and developing mutual translation between the real-world and clinical trials to inform clinical study design.

FIGURE 1

Summary description of the SOMO approach. Data are retrieved and pre‐processes (step a), explorative analysis (step b), estimating the impact of factors on translation between cohorts (step c), validation and evaluation of the results (step d).

FIGURE 2

Overall survival Kaplan–Meier curves and number of censored patients across the time for the (a) baseline survival gap of measurement of outcome (n = 1224), (b) synthetic oversampled RWD cohort with same trial censoring (n = 1992). The cohort in (b) was obtained by oversampling RWD with the SMOTENC algorithm to match RCT sample size, and simulating RWD censoring using the same distribution of the RCT cohort. RWD, real‐world data.

FIGURE 3

Overall Survival Kaplan–Meier Curves using (a) traditional propensity score marching (n = 456), and (b) propensity score accounting trials censoring (n = 456). RWD, real‐world data.

FIGURE 4

Matching simulation results for overall survival hazard ratios for PDS_EliLilly and PDS_Amgen. For each simulation scenario are reported the boxplots of the hazard ratio distribution. BM, brain metastases; PR.SC., propensity score; RWD, real‐world data; Strat.: stratification. Hazard ratio and 95% confidence level interval with the whole cohort is reported in red dotted lines, ideal scenario of RCT‐RWD matching is reported with the blue‐dotted line.

FIGURE 5

Matching simulation results for progress‐free survival hazard ratios for PDS_EliLilly. For each simulation scenario are reported the boxplots of the hazard ratio distribution. BM, Brain metastases; PR.SC., Propensity score; RWD, real‐world data; Strat., Stratification. Hazard ratio and 95% confidence level interval with the whole cohort is reported in red dotted lines, ideal scenario of RCT‐RWD matching is reported with the blue‐dotted line.