Data Cleaning in the Evaluation of a Multi-Site Intervention Project

Abstract

Context: The High Value Healthcare Collaborative (HVHC) sepsis project was a two-year multi-site project where Member health care delivery systems worked on improving sepsis care using a dissemination & implementation framework designed by HVHC. As part of the project evaluation, participating Members provided 5 data submissions over the project period. Members created data files using a uniform specification, but the data sources and methods used to create the data sets differed. Extensive data cleaning was necessary to get a data set usable for the evaluation analysis.

Case description: HVHC was the coordinating center for the project and received and cleaned all data submissions. Submissions received 3 sequentially more detailed levels of checking by HVHC. The most detailed level evaluated validity by comparing values within-Member over time and between Member. For a subset of episodes Member-submitted data were compared to matched Medicare claims data.

Findings: Inconsistencies in data submissions, particularly for length-of-stay variables were common in early submissions and decreased with subsequent submissions. Multiple resubmissions were sometimes required to get clean data. Data checking also uncovered a systematic difference in the way Medicare and some members defined intensive care unit stay.

Conclusions: Data checking is a critical for ensuring valid analytic results for projects using electronic health record data. It is important to budget sufficient resources for data checking. Interim data submissions and checks help find anomalies early. Data resubmissions should be checked as fixes can introduce new errors. Communicating with those responsible for creating the data set provides critical information.

Keywords: data completeness; data error; data quality; data validity; electronic health record; routinely collected health data.

Figure 1

Comparison of the relationship between total length of stay (LOS) and LOS in the intensive care unit (ICU LOS) showing valid and invalid ICU LOS.

Figure 2

Density plots of intensive care unit length of stay (LOS) for selected Members and project quarters showing anomalies for Member 2.

Figure 3

Proportion (95 percent confidence interval) of reported episodes with complete 3-hour sepsis bundle by quarter showing potential anomalies in reporting for Member 3.