Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2024 May 30:8:309.
doi: 10.12688/wellcomeopenres.19565.2. eCollection 2023.

Where next for partial randomisation of research funding? The feasibility of RCTs and alternatives

Tom Stafford  1 , Ines Rombach  1 , Dan Hind  1 , Bilal Mateen  2 , Helen Buckley Woods  3 , Munya Dimario  1 , James Wilsdon  4
Affiliations
Review

Where next for partial randomisation of research funding? The feasibility of RCTs and alternatives

Tom Stafford et al. Wellcome Open Res. .

Abstract

We outline essential considerations for any study of partial randomisation of research funding, and consider scenarios in which randomised controlled trials (RCTs) would be feasible and appropriate. We highlight the interdependence of target outcomes, sample availability and statistical power for determining the cost and feasibility of a trial. For many choices of target outcome, RCTs may be less practical and more expensive than they at first appear (in large part due to issues pertaining to sample size and statistical power). As such, we briefly discuss alternatives to RCTs. It is worth noting that many of the considerations relevant to experiments on partial randomisation may also apply to other potential experiments on funding processes (as described in The Experimental Research Funder's Handbook. RoRI, June 2022).

Keywords: experiments; lottery; metaresearch; metascience; review.

PubMed Disclaimer

Conflict of interest statement

Competing interests: TS and HBW are members of the Research on Research Institute which has received funding from the Wellcome Trust. IR’s post has been funded by income from different NIHR, MRC and charity research funding streams. DH serves as a member of the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Clinical Evaluation and Trials (CET) award committee. His post has been funded by income from different NIHR and MRC research funding streams since 2008. BAM is an employee of the Wellcome Trust, which has funded the Research on Research Institute. MD serves as a member of the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Clinical Evaluation and Trials (CET) award committee. His post has been funded by income from different NIHR and MRC research funding streams. JRW is Director of the Research on Research Institute which has received funding from the Wellcome Trust.

References

    1. Barlösius E, Philipps A: Random grant allocation from the researchers’ perspective: introducing the distinction into legitimate and illegitimate problems in Bourdieu’s field theory. Soc Sci Inf. 2022;61(1):154–178. 10.1177/05390184221076627 - DOI
    1. Bedessem B: Should we fund research randomly? An epistemological criticism of the lottery model as an alternative to peer review for the funding of science. Res Eval. 2020;29(2):150–157. 10.1093/reseval/rvz034 - DOI
    1. Beets MW, von Klinggraeff L, Weaver RG, et al. : Small studies, big decisions: the role of pilot/feasibility studies in incremental science and premature scale-up of behavioral interventions. Pilot Feasibility Stud. 2021;7(1): 173. 10.1186/s40814-021-00909-w - DOI - PMC - PubMed
    1. Bendiscioli S, Firpo T, Bravo-Biosca A, et al. : The experimental research funder’s handbook (Revised edition, June 2022, ISBN 978-1-7397102-0-0). Research on Research Institute. Report,2022. 10.6084/m9.figshare.19459328.v3 - DOI
    1. Bieri M, Roser K, Heyard R, et al. : Face-to-face panel meetings versus remote evaluation of fellowship applications: simulation study at the Swiss National Science Foundation. BMJ open. 2021;11(5): e047386. 10.1136/bmjopen-2020-047386 - DOI - PMC - PubMed