Online physician ratings fail to predict actual performance on measures of quality, value, and peer review

Abstract

Objective: Patients use online consumer ratings to identify high-performing physicians, but it is unclear if ratings are valid measures of clinical performance. We sought to determine whether online ratings of specialist physicians from 5 platforms predict quality of care, value of care, and peer-assessed physician performance.

Materials and methods: We conducted an observational study of 78 physicians representing 8 medical and surgical specialties. We assessed the association of consumer ratings with specialty-specific performance scores (metrics including adherence to Choosing Wisely measures, 30-day readmissions, length of stay, and adjusted cost of care), primary care physician peer-review scores, and administrator peer-review scores.

Results: Across ratings platforms, multivariable models showed no significant association between mean consumer ratings and specialty-specific performance scores (β-coefficient range, -0.04, 0.04), primary care physician scores (β-coefficient range, -0.01, 0.3), and administrator scores (β-coefficient range, -0.2, 0.1). There was no association between ratings and score subdomains addressing quality or value-based care. Among physicians in the lowest quartile of specialty-specific performance scores, only 5%-32% had consumer ratings in the lowest quartile across platforms. Ratings were consistent across platforms; a physician's score on one platform significantly predicted his/her score on another in 5 of 10 comparisons.

Discussion: Online ratings of specialist physicians do not predict objective measures of quality of care or peer assessment of clinical performance. Scores are consistent across platforms, suggesting that they jointly measure a latent construct that is unrelated to performance.

Conclusion: Online consumer ratings should not be used in isolation to select physicians, given their poor association with clinical performance.

Figure 1.

(A) Scatterplots and Lowess line smoother overlays depicting mean consumer ratings scores and mean specialty-specific physician performance scores.

Figure 1.

(B) Scatterplots and Lowess line smoother overlays depicting mean consumer ratings scores and mean PCP scores.

Figure 1.

(C) Scatterplots and Lowess line smoother overlays depicting mean consumer ratings scores and mean administrator scores.

Figure 2.

Scatterplots and linear regression overlays comparing mean consumer ratings scores across platforms.