A demonstration of the impact of response bias on the results of patient satisfaction surveys

Abstract

Objectives: The purposes of the present study were to examine patient satisfaction survey data for evidence of response bias, and to demonstrate, using simulated data, how response bias may impact interpretation of results.

Data sources: Patient satisfaction ratings of primary care providers (family practitioners and general internists) practicing in the context of a group-model health maintenance organization and simulated data generated to be comparable to the actual data.

Study design: Correlational analysis of actual patient satisfaction data, followed by a simulation study where response bias was modeled, with comparison of results from biased and unbiased samples.

Principal findings: A positive correlation was found between mean patient satisfaction rating and response rate in the actual patient satisfaction data. Simulation results suggest response bias could lead to overestimation of patient satisfaction overall, with this effect greatest for physicians with the lowest satisfaction scores.

Conclusions: Findings suggest that response bias may significantly impact the results of patient satisfaction surveys, leading to overestimation of the level of satisfaction in the patient population overall. Estimates of satisfaction may be most inflated for providers with the least satisfied patients, thereby threatening the validity of provider-level comparisons.

Figure 1

Mean Satisfaction Rating by Response Rate by Provider. A circle represents one provider. If there is more than one provider at a single point, the number of lines extending out from the center of the circle equals the total number of providers at that point

Figure 2

Differences between Full Sample Mean Ratings and Biased Mean Ratings by Provider