A tutorial on methods to estimating clinically and policy-meaningful measures of treatment effects in prospective observational studies: a review

Abstract

In randomized controlled trials (RCTs), treatment assignment is unconfounded with baseline covariates, allowing outcomes to be directly compared between treatment arms. When outcomes are binary, the effect of treatment can be summarized using relative risks, absolute risk reductions and the number needed to treat (NNT). When outcomes are time-to-event in nature, the effect of treatment on the absolute reduction of the risk of an event occurring within a specified duration of follow-up and the associated NNT can be estimated. In observational studies of the effect of treatments on health outcomes, treatment is frequently confounded with baseline covariates. Regression adjustment is commonly used to estimate the adjusted effect of treatment on outcomes. We highlight several limitations of measures of treatment effect that are directly obtained from regression models. We illustrate how both regression-based approaches and propensity-score based approaches allow one to estimate the same measures of treatment effect as those that are commonly reported in RCTs. The CONSORT statement recommends that both relative and absolute measures of treatment effects be reported for RCTs with dichotomous outcomes. The methods described in this paper will allow for similar reporting in observational studies.

Keywords: absolute risk reduction; causal effects; confounding; non-randomized studies; number needed to treat; observational studies; odds ratio; propensity score; propensity-score matching; randomized controlled trials; regression; relative risk reduction; survival time; treatment effects.

Figure 1.

Kaplan–Meier survival curves in original, matched and weighted samples