A basic study design for expedited safety signal evaluation based on electronic healthcare data

Abstract

Active drug safety monitoring based on longitudinal electronic healthcare databases (a Sentinel System), as outlined in recent FDA-commissioned reports, consists of several interlocked processes, including signal generation, signal strengthening, and signal evaluation. Once a signal of a potential drug safety issue is generated, signal strengthening and signal evaluation have to follow in short sequence in order to quickly provide as much information about the triggering drug-event association as possible. This paper proposes a basic study design based on the incident user cohort design for expedited signal evaluation in longitudinal healthcare databases. It will not resolve all methodological issues nor will it fit all study questions arising within the framework of a Sentinel System. It should rather be seen as a guidance that will fit the majority of situations and serve as a starting point for adaptations to specific studies. Such an approach will expedite and structure the process of study development and highlight specific assumptions, which is particularly valuable in a Sentinel System where signals are by definition preliminary and evaluation of signals is time critical.

Figure 1

A basic incident user cohort design in longitudinal health care databases. A fixed-length covariate assessment period precedes the initiation of exposure and serves as a washout period of any earlier exposures. Follow-up starts after exposure status is defined

Figure 2

Study design choice by source of exposure variation. The level of drug exposure variation determines the study design choices available for assessing an exposure-event association

Figure 3

Time-varying hazards: the risk of an adverse outcome increases or decreases as a function of time since first use. The shape of a time-varying hazard function depends on the type of event and is caused by a combination of the underlying biology and changes in cohort composition over time

Figure 4

Newly marketed medications and the advantage of the new user design. Medications that were marketed a while ago have reached equilibrium with many prevalent users and few new users while newly marketed drugs by definition will have a high proportion of new users initially until market equilibrium is reached. Comparing incident users of both treatment groups will avoid unfair comparisons of mixed prevalent/incident user cohorts

Figure 5

Dealing with medication switchers. In some chronic conditions, it is impractical to study new users of second-line treatment because these patients are switchers from a first-line agent. The extension of the incident users design (a) is to compare new switchers from a common first-line treatment to the study drug with new switchers to the comparison drug (b). This will preserve the main advantages of the incident user design. Stepping-up therapy can be studied analogously (c)

Figure 6

(a) Achieving balanced cohorts. Matching on the estimated propensity for treatment initiation based on observed patient characteristics will lead to substantially improved balance between treatment groups. Here, propensity score distributions [0,1] are separately plotted for both treatment groups. (b) After successful propensity score matching patient characteristics are balanced for those patients who fall in the area of observed clinical equipoise

Figure 7

Basic design sensitivity analyses. Several sensitivity analyses are recommended to explore the robustness of results, considering limitations inherent in longitudinal healthcare databases

Figure 8

Time-varying exposure. Patients may discontinue their study medication and either change their exposure status to the comparison drug or neither. Such exposure changes are often informed by treatment failure or perceived side effects

Figure 9

Sensitivity analysis of residual confounding. Residual confounding is easiest explored in the array approach or the rule out approach as a function of several parameters that may be informed by empirically derived values to varying degrees: ARR = apparent (observed) relative risk; RR_adjusted = fully adjusted; RR_CD = association between unobserved confounder and disease outcome; OR_EC = association between unobserved confounder and drug exposure; P_C0 = prevalence of confounder in unexposed (C1 = exposed). (a) Array approach based on structural assumptions. Plotted is the fully adjusted RR given a range of structural assumptions for the unobserved confounder. (b) Rule out approach based on specific study findings. The surface to the right and above the plotted curve includes the parameter constellations that would be necessary to fully explain the apparent RR by residual confounding. Figures from reference

Figure A1

Trouble with covariate assessment in case-control studies embedded in longitudinal claims data