Application of a propensity score approach for risk adjustment in profiling multiple physician groups on asthma care

Abstract

Objectives: To develop a propensity score-based risk adjustment method to estimate the performance of 20 physician groups and to compare performance rankings using our method to a standard hierarchical regression-based risk adjustment method.

Data sources/study setting: Mailed survey of patients from 20 California physician groups between July 1998 and February 1999.

Study design: A cross-sectional analysis of physician group performance using patient satisfaction with asthma care. We compared the performance of the 20 physician groups using a novel propensity score-based risk adjustment method. More specifically, by using a multinomial logistic regression model we estimated for each patient the propensity scores, or probabilities, of having been treated by each of the 20 physician groups. To adjust for different distributions of characteristics across groups, patients cared for by a given group were first stratified into five strata based on their propensity of being in that group. Then, strata-specific performance was combined across the five strata. We compared our propensity score method to hierarchical model-based risk adjustment without using propensity scores. The impact of different risk-adjustment methods on performance was measured in terms of percentage changes in absolute and quintile ranking (AR, QR), and weighted kappa of agreement on QR.

Results: The propensity score-based risk adjustment method balanced the distributions of all covariates among the 20 physician groups, providing evidence for validity. The propensity score-based method and the hierarchical model-based method without propensity scores provided substantially different rankings (75 percent of groups differed in AR, 50 percent differed in QR, weighted kappa=0.69).

Conclusions: We developed and tested a propensity score method for profiling multiple physician groups. We found that our method could balance the distributions of covariates across groups and yielded substantially different profiles compared with conventional methods. Propensity score-based risk adjustment should be considered in studies examining quality comparisons.

Figure A1

Stratification by Five Strata Based on Propensity Quintiles

Figure A2

Example of Stratification by Group to Which They Belonged and by Satisfaction with Care

Figure 1

Distributions of Group-Specific Averages of Patient Age, Asthma Severity, and SF-36 PCS among the 20 Physician Groups before Adjustment (Numbers Are Plain Averages within Physician Groups) and after Adjustment (Numbers Are Described in Paragraph 3 of Results) with the Propensity Scores

Figure 2

Ranking Shift between Hierarchical Outcome Regression Adjustment and Propensity Score-Based Risk Adjustment