Early warnings and repayment plans: novel trial management methods for monitoring and managing data return rates in a multi-centre phase III randomised controlled trial with paper Case Report Forms

Abstract

Background: Monitoring and managing data returns in multi-centre randomised controlled trials is an important aspect of trial management. Maintaining consistently high data return rates has various benefits for trials, including enhancing oversight, improving reliability of central monitoring techniques and helping prepare for database lock and trial analyses. Despite this, there is little evidence to support best practice, and current standard methods may not be optimal.

Methods: We report novel methods from the Trial of Imaging and Schedule in Seminoma Testis (TRISST), a UK-based, multi-centre, phase III trial using paper Case Report Forms to collect data over a 6-year follow-up period for 669 patients. Using an automated database report which summarises the data return rate overall and per centre, we developed a Microsoft Excel-based tool to allow observation of per-centre trends in data return rate over time. The tool allowed us to distinguish between forms that can and cannot be completed retrospectively, to inform understanding of issues at individual centres. We reviewed these statistics at regular trials unit team meetings. We notified centres whose data return rate appeared to be falling, even if they had not yet crossed the pre-defined acceptability threshold of an 80% data return rate. We developed a set method for agreeing targets for gradual improvement with centres having persistent data return problems. We formalised a detailed escalation policy to manage centres who failed to meet agreed targets. We conducted a post-hoc, descriptive analysis of the effectiveness of the new processes.

Results: The new processes were used from April 2015 to September 2016. By May 2016, data return rates were higher than they had been at any time previously, and there were no centres with return rates below 80%, which had never been the case before. In total, 10 centres out of 35 were contacted regarding falling data return rates. Six out of these 10 showed improved rates within 6-8 weeks, and the remainder within 4 months.

Conclusions: Our results constitute preliminary effectiveness evidence for novel methods in monitoring and managing data return rates in randomised controlled trials. We encourage other researchers to work on generating better evidence-based methods in this area, whether through more robust evaluation of our methods or of others.

Keywords: Case Report Form returns; Central monitoring; Data completeness; Data management; Data return rates; Trial management.

Fig. 1

Summary of general processes for data collection, data cleaning and data quality assurance in TRISST. Note that the detail in this figure is presented to give context of data management in TRISST. The principle focus of the current work is in the ‘Management’ row, specifically how to most usefully review and act on data about trial data completeness. MRC CTU at UCL Medical Research Council Clinical Trials Unit at University College London, TRISST Trial of Imaging and Schedule in Seminoma Testis

Fig. 2

Changes in data return rates over the course of TRISST. a Overall data return rate (DRR) as reported at each Trial Management Group meeting. b Proportion of all centres with <80% overall DRR (number of centres given above each column); note that the reduction in proportion in the first years of the trial was mainly due to increasing numbers of centres participating in the trial. c Overall trial recruitment and per-month number of Case Report Forms expected, for context. Shaded area shows the time when the new methods, described in this article, were used. TRISST Trial of Imaging and Schedule in Seminoma Testis

Fig. 3

Summary of TRISST data return rate monitoring methods and supporting systems

Fig. 4

a Screenshot of automated form status report. b Screenshot of Excel-based data return rate trend tool. CRF Case Report Form, MRC Medical Research Council, TRISST Trial of Imaging and Schedule in Seminoma Testis

Fig. 5

Data return rates of centres contacted regarding falling data return rates between 13 May 2015 and 28 September 2016. Thick black line in each plot indicates the 80% acceptability threshold. Marker on each line is the date of the team meeting at which it was agreed to contact the centre