Influences of hospital information systems, indicator data collection and computation on reported Dutch hospital performance indicator scores

Abstract

Background: For health care performance indicators (PIs) to be reliable, data underlying the PIs are required to be complete, accurate, consistent and reproducible. Given the lack of regulation of the data-systems used in the Netherlands, and the self-report based indicator scores, one would expect heterogeneity with respect to the data collection and the ways indicators are computed. This might affect the reliability and plausibility of the nationally reported scores.

Methods: We aimed to investigate the extent to which local hospital data collection and indicator computation strategies differ and how this affects the plausibility of self-reported indicator scores, using survey results of 42 hospitals and data of the Dutch national quality database.

Results: The data collection and indicator computation strategies of the hospitals were substantially heterogenic. Moreover, the Hip and Knee replacement PI scores can be regarded as largely implausible, which was, to a great extent, related to a limited (computerized) data registry. In contrast, Breast Cancer PI scores were more plausible, despite the incomplete data registry and limited data access. This might be explained by the role of the regional cancer centers that collect most of the indicator data for the national cancer registry, in a standardized manner. Hospitals can use cancer registry indicator scores to report to the government, instead of their own locally collected indicator scores.

Conclusions: Indicator developers, users and the scientific field need to focus more on the underlying (heterogenic) ways of data collection and conditional data infrastructures. Countries that have a liberal software market and are aiming to implement a self-report based performance indicator system to obtain health care transparency, should secure the accuracy and precision of the heath care data from which the PIs are calculated. Moreover, ongoing research and development of PIs and profound insight in the clinical practice of data registration is warranted.

Figure 1

Reported data infrastructure of the orthopedic and oncology sets" as short and concise descriptive titel of Figure 1ABC. ABC: Reported data infrastructure of the orthopedic and oncology sets. AUT = fully automatic accessible, Partly = partly automatic, partly manually accessible, Man = manually accessible, NOT = not available; HR = Hip replacement, KR = Knee replacement, BC = Mammacarcinoma; Numbers 1, 2, 3 etc. = numbers that indicate the indicator variable which is part of the indicator set; 2b 4b 5b etc. = the unique number of the indicator.