Estimating the effect of COVID-19 on trial design characteristics: a registered report

Abstract

There have been reports of poor-quality research during the COVID-19 pandemic. This registered report assessed design characteristics of registered clinical trials for COVID-19 compared to non-COVID-19 trials to empirically explore the design of clinical research during a pandemic and how it compares to research conducted in non-pandemic times. We did a retrospective cohort study with a 1 : 1 ratio of interventional COVID-19 registrations to non-COVID-19 registrations, with four trial design outcomes: use of control arm, randomization, blinding and prospective registration. Logistic regression was used to estimate the odds ratio of investigating COVID-19 versus not COVID-19 and estimate direct and total effects of investigating COVID-19 for each outcome. The primary analysis showed a positive direct and total effect of COVID-19 on the use of control arms and randomization. It showed a negative direct effect of COVID-19 on blinding but no evidence of a total effect. There was no evidence of an effect on prospective registration. Taken together with secondary and sensitivity analyses, our findings are inconclusive but point towards a higher prevalence of key design characteristics in COVID-19 trials versus controls. The findings do not support much existing COVID-19 research quality literature, which generally suggests that COVID-19 led to a reduction in quality. Limitations included some data quality issues, minor deviations from the pre-registered plan and the fact that trial registrations were analysed which may not accurately reflect study design and conduct. Following in-principle acceptance, the approved stage 1 version of this manuscript was pre-registered on the Open Science Framework at https://doi.org/10.17605/OSF.IO/5YAEB. This pre-registration was performed prior to data analysis.

Keywords: 2019-nCoV; SARS-CoV-2; coronavirus; infectious disease; observational; risk of bias.

Figure 1.

DAG [39] showing assumptions regarding relationships between variables in our study. Nodes represent variables for which we collected data, and edges represent the assumed causal pathways between variables. All mediator variables have been included in a single node for clarity of presentation (other covariates). Sponsor type is a confounder. ‘COVID-19’ means studying COVID-19 as the indication in a trial and is the exposure. ‘Outcome’ is one of the use of control arm, randomization, blinding or prospective registration; ‘Other covariates’ are source registry, phase, geographical region(s), multi- versus single-centre, primary purpose, sample size and intervention type(s). We have not included some relationships between ‘Other covariates’ because they do not impact interpretation of the DAG. For example, phase of the trial likely influences whether the trial is multi- versus single-centre.

Figure 2.

Flow of data through the study. COVID export refers to the dataset of COVID-19 trials maintained on the WHO website (https://www.who.int/clinical-trials-registry-platform). Full ICTRP export refers to the full data download available from ICTRP (https://www.who.int/clinical-trials-registry-platform). IM = indication-matched.

Figure 3.

COVID-19 trial characteristics over time (N = 818 in total, 23 with missing start date excluded from the figure).

Figure 4.

Key results figure. Results presented are odds ratios estimated using logistic regression with 95% confidence intervals (Bonferroni corrected). Details of adjustment sets are given in the methods. Numbers in brackets correspond to the analysis numbers given in the methods for ease of reference. ‘Main direct’ (analysis 4) is the primary analysis. p-values and specific values for the odds ratio and confidence interval are given in table 5. IM = indication-matched.

Figure 5.

Analyses 4 to 7 repeated without trials for which outcome data were missing or not applicable (analysis 10). A table with coefficient, confidence interval and p-values is provided in the GitHub repository.