Comparing the agreement among alternative models in evaluating HMO efficiency

Abstract

Objective: To describe the efficiency of HMOs and to test the robustness of these findings across alternative models of efficiency. This study examines whether these models, when constructed in parallel to use the same information, provide researchers with the same insights and identify the same trends.

Data sources: A data set containing 585 HMOs operating from 1985 through 1994. Variables include enrollment, utilization, and financial information compiled primarily from Health Care Investment Analysts, InterStudy HMO Census, and Group Health Association of America.

Study design: We compute three estimates of efficiency for each HMO and compare the results in terms of individual performance and industry-wide trends. The estimates are then regressed against measures of case mix, quality, and other factors that may be related to the model estimates.

Principal findings: The three models identify similar trends for the HMO industry as a whole; however, they assess the relative technical efficiency of individual firms differently. Thus, these techniques are limited for either benchmarking or setting rates because the firms identified as efficient may be a consequence of model selection rather than actual performance.

Conclusions: The estimation technique to evaluate efficient firms can affect the findings themselves. The implications are relevant not only for HMOs, but for efficiency analyses in general. Concurrence among techniques is no guarantee of accuracy, but it is reassuring; conversely, radically distinct inferences across models can be a warning to temper research conclusions.