Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jun 22;24(1):426.
doi: 10.1186/s13063-023-07348-3.

The small trial problem

Jean Raymond  1 , Tim E Darsaut  2 , Johanna Eneling  3 , Miguel Chagnon  4
Affiliations

The small trial problem

Jean Raymond et al. Trials. .

Abstract

Background: Many randomized trials that aim to assess new or commonly used medical or surgical interventions have been so small that the validity of conclusions becomes questionable.

Methods: We illustrate the small trial problem using the power calculation of five Cochrane-reviewed studies that compared vertebroplasty versus placebo interventions. We discuss some of the reasons why the statistical admonition not to dichotomize continuous variables may not apply to the calculation of the number of patients required for trials to be meaningful.

Results: Placebo-controlled vertebroplasty trials planned to recruit between 23 and 71 patients per group. Four of five studies used the standardized mean difference of a continuous pain variable (centimeters on the visual analog scale (VAS)) to plan implausibly small trials. What is needed is not a mean effect at the population level but a measure of efficacy at the patient level. Clinical practice concerns the care of individual patients that vary in many more respects than the variation around the mean of a single selected variable. The inference from trial to practice concerns the frequency of success of the experimental intervention performed one patient at a time. A comparison of the proportions of patients reaching a certain threshold is a more meaningful method that appropriately requires larger trials.

Conclusion: Most placebo-controlled vertebroplasty trials used comparisons of means of a continuous variable and were consequently very small. Randomized trials should instead be large enough to account for the diversity of future patients and practices. They should offer an evaluation of a clinically meaningful number of interventions performed in various contexts. Implications of this principle are not specific to placebo-controlled surgical trials. Trials designed to inform practice require a per-patient comparison of outcomes and the size of the trial should be planned accordingly.

Keywords: Dichotomization; Number of patients; Placebo-controlled surgical trials; Pragmatic trials; Problems with continuous variables; Sample size; Surgery; Trial methodology; Trial size.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

References

    1. Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. Lancet. 2005;365:1348–1353. doi: 10.1016/S0140-6736(05)61034-3. - DOI - PubMed
    1. Wartolowska K, Judge A, Hopewell S, Collins GS, Dean BJ, Rombach I, et al. Use of placebo controls in the evaluation of surgery: systematic review. BMJ. 2014;348:g3253. doi: 10.1136/bmj.g3253. - DOI - PMC - PubMed
    1. Brown DL, Redberg RF. Last nail in the coffin for PCI in stable angina? Lancet. 2018;391:3–4. doi: 10.1016/S0140-6736(17)32757-5. - DOI - PubMed
    1. Bhatt DL, Gersh BJ, Steg PG, Harrington RA, Windecker S. Rediscovering the orbit of percutaneous coronary intervention after ORBITA. Circulation. 2018;137:2427–2429. doi: 10.1161/CIRCULATIONAHA.118.034596. - DOI - PubMed
    1. Munk PL, Liu DM, Murphy KP, Baerlocher MO. Effectiveness of vertebroplasty: a recent controversy. Can Assoc Radiol J. 2009;60:170–171. doi: 10.1016/j.carj.2009.08.001. - DOI - PubMed