Differences in interaction and subgroup-specific effects were observed between randomized and nonrandomized studies in three empirical examples

Abstract

Objective: To determine the comparability of subgroup-specific and interaction effects (differences between subgroups) between different study designs.

Study design and setting: We compared effects of interventions based on observational studies, randomized clinical trials (RCTs), and individual patient data meta-analyses (IPDMAs) of RCTs (reference) on three clinical topics: (1) mammography screening and breast cancer mortality, (2) coronary artery bypass surgery (CABG) and all-cause mortality, and (3) statins and incidence of major coronary events. Main, subgroup-specific, and interaction effects were compared.

Results: Main and subgroup-specific effects were comparable with respect to the direction of the effects. Differences in the magnitude of subgroup-specific effects in observational studies yielded different interactions compared with those in IPDMA. In the mammography example, the ratio of risk ratios (RRR) (i.e., interaction effect) among observational studies was 1.46 [95% confidence interval (CI): 1.09, 1.96] compared with an IPDMA effect of 1.10 (95% CI: 0.89, 1.37). For the CABG studies, the observational RRR was 1.03 (95% CI: 0.84, 1.26), whereas in the IPDMA, this was 1.40 (95% CI: 1.08, 1.1.81). Finally, in the statin example, the RRR was 1.35 (95% CI: 1.13, 1.61) and 0.90 (95% CI: 0.84, 0.97) for observational studies and IPDMA, respectively.

Conclusion: Main and subgroup-specific effects based on observational data were similar to main and subgroup-specific effects in IPDMAs based on RCTs, yet interactions differed.