Meaningful variation in performance: what does variation in quality tell us about improving quality?

Abstract

Background: Variance reduction is sometimes considered as a goal of clinical quality improvement. Variance among physicians, hospitals, or health plans has been evaluated as the proportion of total variance (or intraclass correlation, ICC) in a quality measure; low ICCs have been interpreted to indicate low potential for quality improvement at that level. However, the absolute amount of variation, expressed in clinically meaningful units, is less frequently reported. Moreover, changes in variance components have not been studied as quality improves.

Objectives: To examine changes in variance components at primary care physician and medical facility levels as performance improved for 4 quality indicators: systolic blood pressure levels in hypertension; low-density lipoprotein-cholesterol levels in hyperlipidemia; patient-reported care experience scores after primary care visits; and mammography screening rates.

Population: Adult members (n = 62,596-410,976) of Kaiser Permanente in Northern California, served by more than 1000 primary care physicians in 35 facilities, from 2001 to 2006.

Methods: Multilevel linear and logistic regression to examine the interphysician and interfacility variances in 4 quality indicators over 6 years, after case-mix adjustment.

Results: ICCs were low for all 4 indicators at both levels (0.0021-0.086). Nevertheless, variances at both levels were statistically and clinically significant. For systolic blood pressure and the care experience score, interfacility and interphysician variance as well as ICCs decreased further as quality improved; declines were greater at the facility level. For low-density lipoprotein-cholesterol, variability at both levels increased with quality improvement; and for screening mammography, small declines were not statistically significant for either physicians or facilities.

Conclusions: Low proportions of variance do not predict low potential for quality improvement. Despite low ICCs for facilities, quality improvement efforts directed primarily at facilities improved quality for all 4 indicators.