Task-independent metrics to assess the data quality of medical registries using the European Society of Thoracic Surgeons (ESTS) Database

Abstract

Objective: The quality of data collected into a database is of paramount importance in every analysis. No standardized methods are available to quantify the quality of data in medical registries. Expanding the work done in other fields, we aimed at developing a methodological approach to measure the quality of a thoracic surgical database, by using the European Society of Thoracic Surgeons (ESTS) Database.

Methods: A selection of anonymized data collected in the ESTS Database from 2007 to 2009 was tested using appropriate data quality metrics: completeness, correctness, consistency and believability. Particularly, the believability value is obtained as a result of a min-max operation based on the evaluation of completeness, correctness and consistency. Completeness measures the number of missing values in each checked column of the database, and it is calculated as number of variables registered/number of variables expected. Correctness reflects the number of data units in error referring to a set of clearly defined criteria (number of correct data/number of all data counted) and consistency is calculated by verifying the number of data in conflict in the same recorded patient (number of consistent checks/total number of checks). The threshold selected to indicate good quality was 0.8.

Results: A total of 49363 values were reviewed to obtain the quality indicators. The results of the data quality assessment for the analyzed section of the ESTS Database are all above the predefined threshold: completeness is 0.85, correctness 0.99 and consistency 0.98. The believability score of data in the database is 0.85.

Conclusions: We were able to apply task-independent metrics to measure the quality within the ESTS Database. This study may represent a template to be applied in the medical/surgical field to test the quality of data in clinical registries. Only registries with a high quality of data can be reliably used for scientific, managerial and credentialing purposes.