Estimation of treatment effects in observational stroke care data: comparison of statistical approaches

Abstract

Introduction: Various statistical approaches can be used to deal with unmeasured confounding when estimating treatment effects in observational studies, each with its own pros and cons. This study aimed to compare treatment effects as estimated by different statistical approaches for two interventions in observational stroke care data.

Patients and methods: We used prospectively collected data from the MR CLEAN registry including all patients (n = 3279) with ischemic stroke who underwent endovascular treatment (EVT) from 2014 to 2017 in 17 Dutch hospitals. Treatment effects of two interventions - i.e., receiving an intravenous thrombolytic (IVT) and undergoing general anesthesia (GA) before EVT - on good functional outcome (modified Rankin Scale ≤2) were estimated. We used three statistical regression-based approaches that vary in assumptions regarding the source of unmeasured confounding: individual-level (two subtypes), ecological, and instrumental variable analyses. In the latter, the preference for using the interventions in each hospital was used as an instrument.

Results: Use of IVT (range 66-87%) and GA (range 0-93%) varied substantially between hospitals. For IVT, the individual-level (OR ~ 1.33) resulted in significant positive effect estimates whereas in instrumental variable analysis no significant treatment effect was found (OR 1.11; 95% CI 0.58-1.56). The ecological analysis indicated no statistically significant different likelihood (β = - 0.002%; P = 0.99) of good functional outcome at hospitals using IVT 1% more frequently. For GA, we found non-significant opposite directions of points estimates the treatment effect in the individual-level (ORs ~ 0.60) versus the instrumental variable approach (OR = 1.04). The ecological analysis also resulted in a non-significant negative association (0.03% lower probability).

Discussion and conclusion: Both magnitude and direction of the estimated treatment effects for both interventions depend strongly on the statistical approach and thus on the source of (unmeasured) confounding. These issues should be understood concerning the specific characteristics of data, before applying an approach and interpreting the results. Instrumental variable analysis might be considered when unobserved confounding and practice variation is expected in observational multicenter studies.

Keywords: Acute ischemic stroke; Confounding by indication; Ecological-analysis; General anesthesia; Instrumental variable; Intravenous thrombolysis; Statistical approaches; Unmeasured confounding.

Fig. 1

Instrumental variable assumptions in relation to estimation of treatment effect on the outcome. U/M Unmeasured/measured confounders

Fig. 2

Differences in the percentage of patients receiving IVT (A) and undergoing general anesthesia (B) before EVT and good functional outcome in 17 EVT hospitals in the Netherlands. Note: Good functional outcome is defined as mRS 0–2 at 90 days

Fig. 3

Effect estimates of receiving IVT intervention on the good functional outcome (mRS 0–2 at 90 days) from four statistical methods of A-1 logistic regression, A-2 generalized estimating equation, B ecological analysis, and C instrumental variable analysis. ^# Case-mix variables in the models are including age, sex, medical history, NIHSS score baseline, and time from onset to arrival at the ED of intervention hospital. Hospital volume was also added to the instrumental variable analysis. * Difference in the absolute probability of a good functional outcome for every 1% of the cases receiving IVT before EVT. For example, the unadjusted coefficient implies that the absolute probability of a good functional outcome is 0.15% higher for a patient treated at a hospital utilizing IVT intervention in 1% of the cases compared with one not utilizing the intervention

Fig. 4

Effect estimates of undergoing general anesthesia on the good functional outcome (mRS 0–2 at 90 days) from four statistical methods of A-1 logistic regression, A-2 generalized estimating equation, B ecological analysis, and C instrumental variable analysis. ^# Case-mix variables in the models are including age, sex, medical history, NIHSS score baseline, and time from onset to arrival at the ED of intervention hospital. Hospital volume was also added to the instrumental variable analysis. * Difference in the absolute probability of a good functional outcome for every 1% of the cases receiving general anesthesia. For example, the unadjusted coefficient implies that the absolute probability of a good functional outcome is 0.05% higher for a patient treated at a hospital utilizing general anesthesia in 1% of the cases compared with one not utilizing the intervention