Benchmarking with Spine Tango: potentials and pitfalls

Abstract

The newly released online statistics function of Spine Tango allows comparison of own data against the aggregated results of the data pool that all other participants generate. This comparison can be considered a very simple way of benchmarking, which means that the quality of what one organization does is compared with other similar organizations. The goal is to make changes towards better practice if benchmarking shows inferior results compared with the pool. There are, however, pitfalls in this simplified way of comparing data that can result in confounding. This means that important influential factors can make results appear better or worse than they are in reality and these factors can only be identified and neutralized in a multiple regression analysis performed by a statistical expert. Comparing input variables, confounding is less of a problem than comparing outcome variables. Therefore, the potentials and limitations of automated online comparisons need to be considered when interpreting the results of the benchmarking procedure.

Fig. 1

Raw and unsorted proportions of dura lesions in seven selected Spine Tango hospitals

Fig. 2

Raw but sorted proportions of dura lesions in seven selected Spine Tango hospitals

Fig. 3

Raw and sorted proportions of dura lesions with standard errors in seven selected Spine Tango hospitals. In addition, the average raw proportion of dura lesions as raw benchmark is displayed for the seven hospitals

Fig. 4

Unadjusted and sorted probabilities of dura lesions with standard errors in seven selected Spine Tango hospitals. In addition, the average probability of dura lesions as raw benchmark is displayed for the seven hospitals

Fig. 5

Unadjusted but stratified (four outcomes) and sorted probabilities of dura lesions with standard errors in seven selected Spine Tango hospitals. In addition, the overall average probability of dura lesions (global benchmark) and the stratum specific average probability of dura lesions as stratum specific benchmark is displayed for the seven hospitals

Fig. 6

Graphical visualization of the mathematical step for calculating a non-existing outcome level for dura lesions by number of segments (3–4 levels). Calculation and display of average (adjusted benchmark) and individual adjusted probability of dura lesions for the seven hospitals

Fig. 7

Contrasting the raw display of proportions of dura lesions with the adjusted probabilities and adjusted benchmark of dura lesions