Conducting Real-world Evidence Studies on the Clinical Outcomes of Diabetes Treatments

Abstract

Real-world evidence (RWE), the understanding of treatment effectiveness in clinical practice generated from longitudinal patient-level data from the routine operation of the healthcare system, is thought to complement evidence on the efficacy of medications from randomized controlled trials (RCTs). RWE studies follow a structured approach. (1) A design layer decides on the study design, which is driven by the study question and refined by a medically informed target population, patient-informed outcomes, and biologically informed effect windows. Imagining the randomized trial we would ideally perform before designing an RWE study in its likeness reduces bias; the new-user active comparator cohort design has proven useful in many RWE studies of diabetes treatments. (2) A measurement layer transforms the longitudinal patient-level data stream into variables that identify the study population, the pre-exposure patient characteristics, the treatment, and the treatment-emergent outcomes. Working with secondary data increases the measurement complexity compared to primary data collection that we find in most RCTs. (3) An analysis layer focuses on the causal treatment effect estimation. Propensity score analyses have gained in popularity to minimize confounding in healthcare database analyses. Well-understood investigator errors, like immortal time bias, adjustment for causal intermediates, or reverse causation, should be avoided. To increase reproducibility of RWE findings, studies require full implementation transparency. This article integrates state-of-the-art knowledge on how to conduct and review RWE studies on diabetes treatments to maximize study validity and ultimately increased confidence in RWE-based decision making.

Keywords: Diabetes; bias; causal treatment effects; confounding; healthcare databases; measurement; pharmacoepidemiology; real-world evidence; regulatory decisions.

Graphical Abstract

Figure 1.

Data sources used by RWE studies. EHR, electronic health records; NDI, National Death Index; PRO, patient-reported outcomes; Adapted from: Franklin JM, Glynn RJ, Martin D, Schneeweiss S. Evaluating the use of nonrandomized real world data analyses for regulatory decision making. Clin Pharm Ther 2019;105:867–77.

Figure 2.

Contributions of RWE for regulatory and coverage decision-making. Adapted from: Franklin JM, Glynn RJ, Martin D, Schneeweiss S. Evaluating the use of nonrandomized real world data analyses for regulatory decision making. Clin Pharm Ther 2019;105:867–77.

Figure 3.

From longitudinal electronic healthcare data to a causal cohort study design. Adapted from: Schneeweiss S. Automated data-adaptive analytics for electronic healthcare data to study causal treatment effects. Clin Epidemiol, 2018:10;771–88.

Figure 4.

Illustration of typical longitudinal study design choices in pharmacoepidemiology. The diagrams use a comparative analysis of new users of pioglitazone versus new users of rosiglitazone on some health outcomes to illustrate how time windows are used to identify key markers and variables.

Figure 5.

From real-world data to real-world evidence.

Figure 6.

The study question and sources of treatment exposure variation guide design choices. Adapted from: Schneeweiss S. A basic study design for expedited safety signal evaluation based on electronic healthcare data. Pharmaceopidemiol Drug Safety 2010;19:858–68.

Figure 7.

Schematic of a parallel group RCT and the corresponding cohort design.

Figure 8.

The prediction of the CAROLINA RCT by a RWE study.

Figure 9.

Immortal time bias in diabetes RWE studies illustrated in 3 hypothetical patients. T2DM, type 2 diabetes mellites; SGLT-2i, sodium-glucose transporter-2 inhibitors; oGLD, other glucose-lowering drugs

Figure 10.

Checking balance of unmeasured pre-exposure covariates in a subsample of patients with enriched clinical data. EHR, electronic health record.

Figure 11.

A flow diagram of considerations for a typical RWE study using healthcare databases. Modified after Schneeweiss S. A basic study design for expedited safety signal evaluation based on electronic healthcare data. Pharmaceopidemiol Drug Safety 2010;19:858–68.

Figure 11.

A flow diagram of considerations for a typical RWE study using healthcare databases. Modified after Schneeweiss S. A basic study design for expedited safety signal evaluation based on electronic healthcare data. Pharmaceopidemiol Drug Safety 2010;19:858–68.