Reproducibility in pharmacometrics applied in a phase III trial of BCG-vaccination for COVID-19

Abstract

Large clinical trials often generate complex and large datasets which need to be presented frequently throughout the trial for interim analysis or to inform a data safety monitory board (DSMB). In addition, reliable and traceability are required to ensure reproducibility in pharmacometric data analysis. A reproducible pharmacometric analysis workflow was developed during a large clinical trial involving 1000 participants over one year testing Bacillus Calmette-Guérin (BCG) (re)vaccination in coronavirus disease 2019 (COVID-19) morbidity and mortality in frontline health care workers. The workflow was designed to review data iteratively during the trial, compile frequent reports to the DSMB, and prepare for rapid pharmacometric analysis. Clinical trial datasets (n = 41) were transferred iteratively throughout the trial for review. An RMarkdown based pharmacometric processing script was written to automatically generate reports for evaluation by the DSMB. Reports were compiled, reviewed, and sent to the DSMB on average three days after the data cut-off, reflecting the trial progress in real-time. The script was also utilized to prepare for the trial pharmacometric analyses. The same source data was used to create analysis datasets in NONMEM format and to support model script development. The primary endpoint analysis was completed three days after data lock and unblinding, and the secondary endpoint analyses two weeks later. The constructive collaboration between clinical, data management, and pharmacometric teams enabled this efficient, timely, and reproducible pharmacometrics workflow.

Figure 1

Overview of the pharmacometric reproducibility workflow. Clinical operations were the initial start of the workflow, but because of its circularity, and ongoing clinical operations throughout the trial parallel to the data processing, the workflow was performed repeatedly. The black boxes represent the clinical team, purple boxes represent the data management team, magenta boxes represent the pharmacometric team, and orange box represent the independent data safety monitoring board (DSMB). All processes were blinded until the end of the clinical trial, after which the data report per arm and the DSMB contained the unblinded data. eCRF = electronic case report form, QC = quality control.

Figure 2

Overview of the data management throughout the clinical trial. Number of participants on trial over time is shown in purple solid line, database locks (n = 2) are shown in black dashed lines, scheduled data review (n = 41) are shown as magenta top rug plot, first reported diagnosed COVID-19 case in South Africa is shown as grey bottom axis mark for reference.

Figure 3

Number of eCRFs submitted per master database. Dataset architecture consisted of 4 master databases (Screening/enrolment, Events, Lab results, and Follow-up) for which the number of records is shown.

Figure 4

Interoperability through standardized file structure and automatic extraction of working directory using the system’s info. The ifelse() statement can be expanded with nested ifelse() statement for more collaborators.

Figure 5

RMarkdown was used to combine text and R variables in the automatically generated report. (A) In-line calling of R variables to include them in a written sentence, (B) R variable CLOSED was used to switch between open and closed reporting using if-statements for tables and graphs called in R-chunks or (C) called in in-line R calls.