Automated multi-instance REDCap data synchronization for NIH clinical trial networks

Abstract

Objectives: The main goal is to develop an automated process for connecting Research Electronic Data Capture (REDCap) instances in a clinical trial network to allow for deidentified transfer of research surveys to cloud computing data commons for discovery.

Materials and methods: To automate the process of consolidating data from remote clinical trial sites into 1 dataset at the coordinating/storage site, we developed a Hypertext Preprocessor script that operates in tandem with a server-side scheduling system (eg, Cron) to set up practical data extraction schedules for each remote site.

Results: The REDCap Application Programming Interface (API) Connection provides a novel implementation for automated synchronization between multiple REDCap instances across a distributed clinical trial network, enabling secure and efficient data transfer between study sites and coordination centers. Additionally, the protocol checker allows for automated reporting on conforming to planned data library protocols.

Discussion: Working from a shared and accepted core library of REDCap surveys was critical to the success of this implementation. This model also facilitates Institutional Review Board (IRB) approvals because the coordinating center can designate which surveys and data elements to be transferred. Hence, protected health information can be transformed or withheld depending on the permission given by the IRB at the coordinating center level. For the NIH HEAL clinical trial networks, this unified data collection works toward the goal of creating a deidentified dataset for transfer to a Gen3 data commons.

Conclusion: We established several simple and research-relevant tools, REDCAP API Connection and REDCAP Protocol Check, to support the emerging needs of clinical trial networks with increased data harmonization complexity.

Keywords: Application Programming Interface; REDCap; chronic pain; opioid use.

Figure 1.

REDCap transform and combination process for multisite data integration. This figure illustrates the automated data transfer workflow between multiple Research Electronic Data Capture (REDCap) instances. The process begins when a scheduled transfer time is reached, triggering a Hypertext Preprocessor (PHP) script to use site-specific Application Programming Interface (API) tokens for executing “Export Records” calls. The exported data are temporarily stored as JavaScript Object Notation (JSON) objects in a cache. These cached objects undergo Extract, Transform, Load (ETL) processing to prepare them for import into the destination REDCap instance. The processed data are then imported into the destination REDCap using its API token through an “Import Records” call. This process repeats according to predefined schedules for each participating site, ensuring systematic and secure data transfer across the research network.

Figure 2.

REDCap API governance structure for network data management. This figure depicts the governance and data flow structure for managing REDCap instances across a clinical research network. The diagram shows how individual study sites connect to the central coordinating center through secure API connections, with specific attention to data access controls and security protocols. The governance structure ensures appropriate data sharing while maintaining institutional security requirements and protecting sensitive information. The system implements scheduled data transfers and maintains proper authentication through API tokens, creating a secure and automated data pipeline between participating institutions and the central coordination hub. Abbreviations: API, Application Programming Interface; REDCap, Research Electronic Data Capture.

Figure 3.

Overview of the Wake Forest NIH HEAL Initiative network programs. This figure presents 2 key National Institutes of Health (NIH) Helping to End Addiction Long-term (HEAL) Initiative programs. The Integrative Management of chronic Pain and Opioid Use Disorder for Whole Recovery (IMPOWR) network focuses on developing and evaluating integrated treatments for patients with comorbid chronic pain and opioid use disorder (OUD). The Multilevel Interventions to Reduce Harm and Improve Quality of Life (MIRHIQL) program supports research aimed at reducing harm and enhancing quality of life for patients on long-term opioid therapy. The diagram illustrates how these programs function within the Wake Forest University School of Medicine REDCap API system, showing the number of participating institutions and the development vs production projects for each network.