Validating a transnational fracture treatment registry using a standardized method

Abstract

Aim: Subsequent to a three-month pilot phase, recruiting patients for the newly established BFCC (Baltic Fracture Competence Centre) transnational fracture registry, a validation of the data quality needed to be carried out, applying a standardized method.

Method: During the literature research, the method of "adaptive monitoring" fulfilled the requirements of the registry and was applied. It consisted of a three-step audit process; firstly, scoring of the overall data quality, followed by source data verification of a sample size, relative to the scoring result, and finally, feedback to the registry on measures to improve data quality. Statistical methods for scoring of data quality and visualisation of discrepancies between registry data and source data were developed and applied.

Results: Initially, the data quality of the registry scored as medium. During source data verification, missing items in the registry, causing medium data quality, turned out to be absent in the source as well. A subsequent adaptation of the score evaluated the registry's data quality as good. It was suggested to add variables to some items in order to improve the accuracy of the registry.

Discussion: The application of the method of adaptive monitoring has only been published by Jacke et al., with a similar improvement of the scoring result following the audit process. Displaying data from the registry in graphs helped to find missing items and discover issues with data formats. Graphically comparing the degree of agreement between the registry and source data allowed to discover systematic faults.

Conclusions: The method of adaptive monitoring gives a substantiated guideline for systematically evaluating and monitoring a registry's data quality and is currently second to none. The resulting transparency of the registry's data quality could be helpful in annual reports, as published by most major registries. As the method has been rarely applied, further successive applications in established registries would be desirable.

Keywords: Data quality; Data validation; Quality assessment; Registry; Scoring.

Fig. 1

Arbitrary example to introduce the statistical method for source data verification. The perception of the hospital personnel (y-axis) versus their actual function (x-axis). Prop = proportions

Fig. 2

Length of stay of all patients in the registry

Fig. 3

Fracture age (in days) of all patients in the registry

Fig. 4

Source data verification on the registered number of comorbidities. Prop = proportions

Fig. 5

Graphical analysis of International Classifications of Diseases (ICD) coding in source data verification. Prop = proportions

Fig. 6

Source data verification of fracture date. Prop = proportions. Instead of “same”, the date 1 January 1900 was selected

Fig. 7

Source data verification (SDV) of reduction method. Prop = proportions

Fig. 8

Occurrence of complications in all patients. Prop = proportions