Effect identification in comparative effectiveness research

Abstract

The widespread adoption of electronic medical records means there are now vast data resources available for comparative effectiveness research (CER). In concert with conventional randomized controlled trials, CER holds great promise for advancing our understanding of how different therapeutic treatments yield different health outcomes in different settings and with different populations. But in a research culture fixated on estimating correlations and p-values, the threat of misinterpretation of results and improper CER inferences is troubling. Accordingly, this paper aims to shore up the inferential foundations of CER by introducing the fundamentals of effect identification, which is the process of identifying or teasing out empirically defensible causal effects from competing explanations. Three primary requirements of effect identification-positivity, exchangeability, and consistency- are explained and simple exampled are given. The take home message is that so-called big data from medical records may not yield better or more useful results. Advances will come only when the right question is addressed with the appropriate data and methods.

Figure 1.

Illustrative plot of fictitious means and 95% confidence intervals, N = 100

Figure 2.

Illustrative plot of fictitious means and 95% confidence intervals, N = 10,000