Observational studies of treatment effectiveness in neurology

Abstract

The capacity and power of data from cohorts, registries and randomized trials to provide answers to contemporary clinical questions in neurology has increased considerably over the past two decades. Novel sophisticated statistical methods are enabling us to harness these data to guide treatment decisions, but their complexity is making appraisal of clinical evidence increasingly demanding. In this review, we discuss several methodological aspects of contemporary research of treatment effectiveness in observational data in neurology, aimed at academic neurologists and analysts specializing in outcomes research. The review discusses specifics of the sources of observational data and their key features. It focuses on the limitations of observational data and study design, as well as statistical approaches aimed to overcome these limitations. Among the examples of leading clinical themes typically studied with analyses of observational data, the review discusses methodological approaches to comparative treatment effectiveness, development of diagnostic criteria and definitions of clinical outcomes. Finally, this review provides a brief summary of key points that will help clinical audience critically evaluate design and analytical aspects of studies of disease outcomes using observational data.

Keywords: causal inference; comparative effectiveness; marginal structural model; methodology; propensity score; statistics.

Figure 1

Directed acyclic graph: an example. The figure illustrates a causal relationship between treatment and outcome, and relationships with other variables (confounder, unmeasured confounder, mediator of treatment effect and collider) that need to be considered in the design of an observational study and its analysis.

Figure 2

From group effects to individual outcomes. Conventional predictive models of outcomes first establish general associations between predictors/exposures and outcomes at the level of populations (left). The outcomes in discrete subgroups (corresponding to different clinical scenarios or demographic strata) are then studied (middle left). It is advantageous to stratify the population based on the relevant predictors/exposures identified in the whole population. To establish discriminative ability of the predictive markers, the individual predictors, their sums and their interactions are studied in patients, accounting for their individual constitution of relevant characteristics (here depicted as ‘a’ and ‘b’; middle right). Accuracy and external validity of the resulting predictive models is established by testing the models in individuals from a non-overlapping (validation) cohort (right).

Figure 3

Prediction of long-term outcomes: a clinical perspective. In conditions with well understood causal relationships such as stroke (A), reliable long-term prognostic models of adverse disease episodes (red arrows) are plausible. These are typically only updated upon change in clinical circumstances. In amnesic conditions such as multiple sclerosis (B–D), predictions of long-term outcomes conducted at a single time point and not updated between adverse disease episodes become increasingly inaccurate over time (B). In this example, the patients’ risk is not updated until the clinical assessment following the next event (blue marker). However, in practice, a clinician would intuitively update the predicted prognosis at every assessment (blue markers), combining the prior prediction with new information (C). This enables the clinician to respond to change in the predicted risk by modifying therapy (green arrow) and potentially preventing the adverse outcome (D). Black lines = predicted risk; white markers = clinical assessments; blue markers = clinical assessment with update of prediction; red arrows = exacerbations/adverse outcomes; green arrow = change of treatment.