Where's the best medicine? The hospital rating game

Abstract

On July 18, 1994, U.S. News and World Report published its annual ranking of America's best hospitals. The rankings were based on a model developed by the National Opinion Research Center (NORC). National rankings are a type of program evaluation that deserve careful scrutiny with regard to their methods. While problems with measurement validity are discussed, the principle concern here was the model used to compile a final ranking of hospitals. The News-NORC model is typical of ranking schemes that usually reduce higher levels of measurement to lower ones in order to compute final scores. Such simple tabulation techniques, however, distort the ratios between the scores and bias the results. A type of percentaging, using part/whole percentages (P/W%), was used to show the differences between hospital rankings when data are kept at higher levels of measurement, such as interval or ratio levels. Percentaging methods produce more valid results when comparatively evaluating programs whether the goal is to produce national rankings or some other outcome.