Weak correlations in health services research: Weak relationships or common error?

Abstract

Objective: To examine whether the correlation between a provider's effect on one population of patients and the same provider's effect on another population is underestimated if the effects for each population are estimated separately as opposed to being jointly modeled as random effects, and to characterize how the impact of the estimation procedure varies with sample size.

Data sources: Medicare claims and enrollment data on emergency department (ED) visits, including patient characteristics, the patient's hospitalization status, and identification of the doctor responsible for the decision to hospitalize the patient.

Study design: We used a three-pronged investigation consisting of analytical derivation, simulation experiments, and analysis of administrative data to demonstrate the fallibility of stratified estimation. Under each investigation method, results are compared between the joint modeling approach to those based on stratified analyses.

Data collection/extraction methods: We used data on ED visits from administrative claims from traditional (fee-for-service) Medicare from January 2012 through September 2015.

Principal findings: The simulation analysis demonstrates that the joint modeling approach is generally close to unbiased, whereas the stratified approach can be severely biased in small samples, a consequence of joint modeling benefitting from bivariate shrinkage and the stratified approach being compromised by measurement error. In the administrative data analyses, the estimated correlation of doctor admission tendencies between female and male patients was estimated to be 0.98 under the joint model but only 0.38 using stratified estimation. The analogous correlations for White and non-White patients are 0.99 and 0.28 and for Medicaid dual-eligible and non-dual-eligible patients are 0.99 and 0.31, respectively. These results are consistent with the analytical derivations.

Conclusions: Joint modeling targets the parameter of primary interest. In the case of population correlations, it yields estimates that are substantially less biased and higher in magnitude than naive estimators that post-process the estimates obtained from stratified models.

Keywords: attenuation; bivariate random effects; bivariate shrinkage; hierarchical model; measurement error.

FIGURE 1

Performance of estimators of ${\rho }_{\mathit{doc}}$ as a function of $\rho$ (Simulation 1), where the sample size per stratum per doctor is 10 (half the total per doctor sample‐size) [Color figure can be viewed at wileyonlinelibrary.com]

FIGURE 2

Performance of estimators of ${\rho }_{\mathit{doc}}$ by the total number of patients per doctor (Simulation), where sample size per stratum per doctor is half the total per doctor sample size and $\rho =0.95$ [Color figure can be viewed at wileyonlinelibrary.com]