The IeDEA harmonist data toolkit: A data quality and data sharing solution for a global HIV research consortium

Abstract

We describe the design, implementation, and impact of a data harmonization, data quality checking, and dynamic report generation application in an international observational HIV research network. The IeDEA Harmonist Data Toolkit is a web-based application written in the open source programming language R, employs the R/Shiny and RMarkdown packages, and leverages the REDCap data collection platform for data model definition and user authentication. The Toolkit performs data quality checks on uploaded datasets, checks for conformance with the network's common data model, displays the results both interactively and in downloadable reports, and stores approved datasets in secure cloud storage for retrieval by the requesting investigator. Including stakeholders and users in the design process was key to the successful adoption of the application. A survey of regional data managers as well as initial usage metrics indicate that the Toolkit saves time and results in improved data quality, with a 61% mean reduction in the number of error records in a dataset. The generalized application design allows the Toolkit to be easily adapted to other research networks.

Keywords: Biomedical informatics; Data harmonization; Data quality; Global health; HIV.

Figure A1.

Each panel tracks the number of errors detected in each iteration of uploading and checking a dataset for a single users’ response to a specific data request. On the final iteration, datasets were transferred to the investigator who requested the data.

Figure 1:

IeDEA regions and participating sites

Figure 2.

Abstracting data model details in REDCap. Data quality checks based on these details automatically include new variables and codes.

Figure 3.

Collaborative design timeline for IeDEA Harmonist Data Toolkit

Figure 4.

Harmonist Data Toolkit system architecture and communication with REDCap

Figure 5.

Report visualizations useful in data quality assessment: (a) Example of histograms of enrollments, clinic visits, lab tests, ART medication initiation, and disease diagnoses by date for each site. Investigators can spot unusual trends, such as the drop off in documented clinic visits after 2015 for this example site. (b) Heat maps of patient representation in data tables (e.g., loss to follow-up from clinic [LTFU], visits, CD4 cell count lab results, HIV viral load lab results, and antiretroviral therapy [ART]) draw attention to gaps in reporting, such as the lack of any clinic visit data from “Site 5” in the example above.

Figure 6.

Harmonist Data Toolkit workflow overview

Figure 7.

Screenshot of Step 2 of the Harmonist Data Toolkit.

Figure 8.

Results of REDCap Survey of IeDEA data managers after the first year of Toolkit use in IeDEA. Data managers compared Toolkit workflow with their previous methods of data quality checking and data sharing for IeDEA multiregional studies.

Figure 9.

Example analysis of dataset errors in a series of uploads and revisions by a single data manager for a specific data request. On the final iteration, the dataset was transferred to the investigator who requested the data. See the Appendix for additional graphs.

Figure 10.

Error types found in initial uploads as compared with final uploads among datasets that were checked with the Toolkit multiple times and revised before submission