Randomized Controlled Trials Studying Nonoperative Treatments of Osteoarthritis Often Use Misleading and Uninformative Control Groups: A Systematic Review

Abstract

Background: Because there are no known treatments that alter the natural course of the pathophysiology of osteoarthritis, nonoperative treatment needs to be compared with known effective treatments that seek to mitigate symptoms or with similarly invasive inert (placebo) treatments to determine effectiveness. Comparing a treatment to an uninformative control group may inappropriately legitimize and support the use of potentially ineffective treatments. We therefore investigated the prevalence of inappropriate control groups in musculoskeletal research and asked whether these are associated with reporting a positive treatment effect.

Questions/purposes: We systematically reviewed randomized trials of nonoperative treatments of osteoarthritis and asked: (1) What proportion of randomized trials use uninformative control groups (defined as a treatment less invasive than the tested treatment, or a treatment that might possibly not outperform placebo but is not acknowledged as such)? (2) Is the use of uninformative control groups independently associated with reporting a positive treatment effect (defined as p < 0.05 in favor of the intervention, or as making a recommendation favoring the intervention over the control treatment)?

Methods: In a systematic review following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we searched PubMed, Cochrane, and Embase up to September 2023 for randomized controlled trials published between 2020 to 2022 that compared one or more nonoperative treatments for the symptoms of osteoarthritis. We excluded studies that contained a surgical treatment group. We identified 103 trials that met eligibility criteria, with a total of 15,491 patients. The risk of bias was high in 60% (n = 62) of trials using the Cochrane Risk of Bias Tool, version 2. Although the high risk of bias in the included studies is concerning, it does not invalidate our design; instead, it highlights that some studies may use flawed methods to recommend treatments with unproven effectiveness beyond nonspecific effects because the kinds of bias observed would tend to increase the apparent benefit of the treatment(s) being evaluated. We used logistic regression to test the association of uninformative control groups with a positive treatment effect, accounting for potential confounders such as conflict of interest and study bias using the Cochrane Risk of Bias score.

Results: The use of uninformative control groups (treatments less invasive than the tested treatment, or treatments that might not outperform placebo but are not acknowledged as such) was found in 46% (47 of 103) of included studies. After accounting for potential confounding, there was no association between reporting positive treatment effects and the use of an uninformative control group. Studies with a low risk of bias had a lower likelihood of reporting a positive treatment effect (OR 0.2 [95% confidence interval 0.05 to 0.9]; p = 0.04, model pseudo R 2 = 0.21).

Conclusion: The finding that recent studies that mimic high-level evidence often use uninformative control groups that do not adequately account for nonspecific effects (perceived treatment benefits unrelated to a treatment's direct physiological effects) points to a high risk of legitimizing ineffective treatments. This raises the ethical imperative for patients, clinicians, journal peer reviewers, and journal editors to hold researchers to the standard of an adequate, informative control group. Awareness and risk of bias checklists might help patients and clinicians forgo new treatments based on seemingly high-level evidence that may carry only iatrogenic, financial, and psychological harm (false hope, in particular).

Level of evidence: Level I, therapeutic study.