Risk-adapted monitoring is not inferior to extensive on-site monitoring: Results of the ADAMON cluster-randomised study

Abstract

Background According to Good Clinical Practice, clinical trials must protect rights and safety of patients and make sure that the trial results are valid and interpretable. Monitoring on-site has an important role in achieving these objectives; it controls trial conduct at trial sites and informs the sponsor on systematic problems. In the past, extensive on-site monitoring with a particular focus on formal source data verification often lost sight of systematic problems in study procedures that endanger Good Clinical Practice objectives. ADAMON is a prospective, stratified, cluster-randomised, controlled study comparing extensive on-site monitoring with risk-adapted monitoring according to a previously published approach. Methods In all, 213 sites from 11 academic trials were cluster-randomised between extensive on-site monitoring (104) and risk-adapted monitoring (109). Independent post-trial audits using structured manuals were performed to determine the frequency of major Good Clinical Practice findings at the patient level. The primary outcome measure is the proportion of audited patients with at least one major audit finding. Analysis relies on logistic regression incorporating trial and monitoring arm as fixed effects and site as random effect. The hypothesis was that risk-adapted monitoring is non-inferior to extensive on-site monitoring with a non-inferiority margin of 0.60 (logit scale). Results Average number of monitoring visits and time spent on-site was 2.1 and 2.7 times higher in extensive on-site monitoring than in risk-adapted monitoring, respectively. A total of 156 (extensive on-site monitoring: 76; risk-adapted monitoring: 80) sites were audited. In 996 of 1618 audited patients, a total of 2456 major audit findings were documented. Depending on the trial, findings were identified in 18%-99% of the audited patients, with no marked monitoring effect in any of the trials. The estimated monitoring effect is -0.04 on the logit scale with two-sided 95% confidence interval (-0.40; 0.33), demonstrating that risk-adapted monitoring is non-inferior to extensive on-site monitoring. At most, extensive on-site monitoring could reduce the frequency of major Good Clinical Practice findings by 8.2% compared with risk-adapted monitoring. Conclusion Compared with risk-adapted monitoring, the potential benefit of extensive on-site monitoring is small relative to overall finding rates, although risk-adapted monitoring requires less than 50% of extensive on-site monitoring resources. Clusters of findings within trials suggest that complicated, overly specific or not properly justified protocol requirements contributed to the overall frequency of findings. Risk-adapted monitoring in only a sample of patients appears sufficient to identify systematic problems in the conduct of clinical trials. Risk-adapted monitoring has a part to play in quality control. However, no monitoring strategy can remedy defects in quality of design. Monitoring should be embedded in a comprehensive quality management approach covering the entire trial lifecycle.

Keywords: Risk-adapted monitoring; clinical trial conduct; clinical trial methodology; on-site monitoring; quality management.

Figure 1.

Profile of the ADAMON study. EM: extensive on-site monitoring; RaM: risk-adapted monitoring.

Figure 2.

Monitoring effect on the primary endpoint. Figure 2 shows the forest plot of a random-effect meta-analysis of within-trial monitoring effects. The overall estimate of the random-effect meta-analysis closely agrees with the model-based estimate of −0.04 with two-sided 95% confidence interval (−0.40; 0.33). There is no significant heterogeneity between trials. Note that trial #05 is non-informative because there were major findings in all but one patient. Trials are grouped by risk class. The intervention effect does not differ by risk class. The black vertical line at 0.6 shows the pre-specified tolerance margin for claiming non-inferiority. Overall and in both subgroups, the non-inferiority margin is outside the meta-analysis confidence interval (CI).

Figure 3.

Model-based estimates of the monitoring effect for the primary patient-level and the secondary error domain–specific finding rates. Error domains: informed consent process (IC), patient selection (eligibility criteria critical for safety and/or efficacy; SEL), intervention (protocol deviation with impact on patient safety or data validity; INTV), endpoint assessment (END) and serious adverse event reporting (SAER). There is no statistical evidence that type of monitoring makes any difference in reducing the number of major findings neither overall nor in specific error domains. Quantitatively, point estimates lie near zero on the logit scale and all two-sided 95% confidence intervals clearly exclude the pre-specified tolerance limit of logit +0.6. CI: confidence interval.