Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Oct 16;19(1):557.
doi: 10.1186/s13063-018-2940-9.

Development of a standardised set of metrics for monitoring site performance in multicentre randomised trials: a Delphi study

Diane Whitham  1 , Julie Turzanski  1 , Lucy Bradshaw  1 , Mike Clarke  2 , Lucy Culliford  3 , Lelia Duley  1 , Lisa Shaw  4 , Zoe Skea  5 , Shaun P Treweek  5 , Kate Walker  1 , Paula R Williamson  6 , Alan A Montgomery  7 ; Site Performance Metrics for Multicentre Randomised Trials Collaboration
Collaborators, Affiliations

Development of a standardised set of metrics for monitoring site performance in multicentre randomised trials: a Delphi study

Diane Whitham et al. Trials. .

Abstract

Background: Site performance is key to the success of large multicentre randomised trials. A standardised set of clear and accessible summaries of site performance could facilitate the timely identification and resolution of potential problems, minimising their impact. The aim of this study was to identify and agree a core set of key performance metrics for managing multicentre randomised trials.

Methods: We used a mixed methods approach to identify potential metrics and to achieve consensus about the final set, adapting methods that are recommended by the COMET Initiative for developing core outcome sets in health care. We used performance metrics identified from our systematic search and focus groups to create an online Delphi survey. We invited respondents to score each metric for inclusion in the final core set, over three survey rounds. Metrics scored as critical by ≥70% and unimportant by <15% of respondents were taken forward to a consensus meeting of representatives from key UK-based stakeholders. Participants in the consensus meeting discussed and voted on each metric, using anonymous electronic voting. Metrics with >50% of participants voting for inclusion were retained.

Results: Round 1 of the Delphi survey presented 28 performance metrics, and a further six were added in round 2. Of 294 UK-based stakeholders who registered for the Delphi survey, 211 completed all three rounds. At the consensus meeting, 17 metrics were discussed and voted on: 15 metrics were retained following survey round 3, plus two others that were preferred by consensus meeting participants. Consensus was reached on a final core set of eight performance metrics in three domains: (1) recruitment and retention, (2) data quality and (3) protocol compliance. A simple tool for visual reporting of the metrics is available from the Nottingham Clinical Trials Unit website.

Conclusions: We have established a core set of metrics for measuring the performance of sites in multicentre randomised trials. These metrics could improve trial conduct by enabling researchers to identify and address problems before trials are adversely affected. Future work could evaluate the effectiveness of using the metrics and reporting tool.

Keywords: Consensus meeting; Delphi survey; Multicentre randomised trials; Performance metrics; Trial management.

PubMed Disclaimer

Conflict of interest statement

Ethics approval and consent to participate

The study was reviewed and approved by the Faculty of Medicine & Health Sciences Research Ethics Committee at the University of Nottingham. Individual participants were approached or self-volunteered to participate in the Delphi survey and consensus meeting. They received a short introductory email with information about the study, including explicit details about what would be involved. Participants were free to withdraw from the study at any time. Study conduct was consistent with standard practice in survey research. Consent was assumed by agreement to participate in meetings and complete online questionnaires.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig. 1
Fig. 1
Summary results of Delphi survey and consensus meeting
Fig. 2
Fig. 2
Worked example of site performance metrics reporting tool in Microsoft Excel. a Summary worksheet, b thresholds worksheet and c trial data worksheet
See this image and copyright information in PMC

References

    1. Dorricott K. Using Metrics to Direct Performance Improvement Efforts in Clinical Trial Management. Monitor. 2012;24(4):9–13.
    1. Blackwood B, Ringrow S, Clarke M, Marshall J, Rose L, Williamson P, et al. Core Outcomes in Ventilation Trials (COVenT): protocol for a core outcome set using a Delphi survey with a nested randomised trial and observational cohort study. Trials. 2015;16:368. doi: 10.1186/s13063-015-0905-9. - DOI - PMC - PubMed
    1. Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. 2011;8(1):e1000393. doi: 10.1371/journal.pmed.1000393. - DOI - PMC - PubMed
    1. Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The COMET Handbook: version 1.0. Trials. 2017;18(Suppl 3):280. doi: 10.1186/s13063-017-1978-4. - DOI - PMC - PubMed
    1. Walker KF, Turzanski J, Whitham D, Montgomery AA, Duley L. Monitoring performance of sites within multicentre randomised trials: A systematic review of performance metrics. Trials. 2018; Accepted for publication. - PMC - PubMed

Publication types

MeSH terms

LinkOut - more resources