Factors explaining the heterogeneity of effects of patient decision aids on knowledge of outcome probabilities: a systematic review sub-analysis

Abstract

Background: There is considerable unexplained heterogeneity in previous meta-analyses of randomized controlled trials (RCTs) evaluating the effects of patient decision aids on the accuracy of knowledge of outcome probabilities. The purpose of this review was to explore possible effect modification by three covariates: the type of control intervention, decision aid quality and patients' baseline knowledge of probabilities.

Methods: A sub-analysis of studies previously identified in the 2011 Cochrane review on decision aids for people facing treatment and screening decisions was conducted. Additional unpublished data were requested from relevant study authors to maximize the number of eligible studies. RCTs (to 2009) comparing decision aids with standardized probability information to control interventions (lacking such information) and assessing the accuracy of patient knowledge of outcome probabilities were included. The proportions of patients with accurate knowledge of outcome probabilities in each group were converted into relative effect measures. Intervention quality was assessed using the International Patient Decision Aid Standards instrument (IPDASi) probabilities domain.

Results: A main effects analysis of 17 eligible studies confirmed that decision aids significantly improve the accuracy of patient knowledge of outcome probabilities (relative risk = 1.80 [1.51, 2.16]), with considerable heterogeneity (87%). The type of control did not modify effects. Meta-regression suggested that the IPDASi probabilities domain score (reflecting decision aid quality) is a potential effect modifier (P = 0.037), accounting for a quarter of the variability (R² = 0.28). Meta-regression indicated the control event rate (reflecting baseline knowledge) is a significant effect modifier (P = 0.001), with over half the variability in ln(OR) explained by the linear relationship with log-odds for the control group (R² = 0.52); this relationship was slightly strengthened after correcting for the statistical dependence of the effect measure on the control event rate.

Conclusions: Patients' baseline level of knowledge of outcome probabilities is an important variable that explains the heterogeneity of effects of decision aids on improving accuracy of this knowledge. Greater relative effects are observed when the baseline proportion of patients with accurate knowledge is lower. This may indicate that decision aids are more effective in populations with lower knowledge.

Figure 1

Main effects of decision aids on patient knowledge of outcome probabilities. CI, confidence interval; df, degrees of freedom; RR, relative risk.

Figure 2

Meta-regression of the effect of decision aid quality: normalized IPDASi probabilities dimension score vs ln(OR). Kuppermann et al. [36] is excluded since this decision aid was not available for scoring on the IPDASi probabilities dimension. The area of each circle is proportional to the weight for that study. IPDASi, International Patient Decision Aid Standards instrument; OR, odds ratio.

Figure 3

Meta-regression of the effect of control event rate: logit control vs ln(OR). The dashed line is prior to bias correction. The solid line is after bias correction. The area of each circle is proportional to the weight for that study. OR, odds ratio.

Figure 4

Empirically fitted relationship predicting relative risk when the control event rate (baseline knowledge) is known. CER, control event rate; RR, relative risk.