Research waste in diagnostic trials: a methods review evaluating the reporting of test-treatment interventions

Abstract

Background: The most rigorous method for evaluating the effectiveness of diagnostic tests is through randomised trials that compare test-treatment interventions: complex interventions comprising episodes of testing, decision-making and treatment. The multi-staged nature of these interventions, combined with the need to relay diagnostic decision-making and treatment planning, has led researchers to hypothesise that test-treatment strategies may be very challenging to document. However, no reviews have yet examined the reporting quality of interventions used in test-treatment RCTs. In this study we evaluate the completeness of intervention descriptions in a systematically identified cohort of test-treatment RCTs.

Methods: We ascertained all test-treatment RCTs published 2004-2007, indexed in CENTRAL. Included trials randomized patients to diagnostic tests and measured patient outcomes after treatment. Two raters examined the completeness of test-treatment intervention descriptions in four components: 1) the test, 2) diagnostic decision-making, 3) management decision-making, 4) treatments.

Results: One hundred and three trials compared 105 control with 119 experimental interventions, most commonly in cardiovascular medicine (35, 34%), obstetrics and gynecology (17%), gastroenterology (14%) or orthopedics (10%). A broad range of tests were evaluated, including imaging (50, 42%), biochemical assays (21%) and clinical assessment (12%). Only five (5%) trials detailed all four components of experimental and control interventions, none of which also provided a complete care pathway diagram. Experimental arms were missing descriptions of tests, diagnostic-decision making, management planning and treatments (36%, 51%, 55% and 79% of trials respectively); control arms were missing the same details in 61%, 66%, 67% and 84% of trials.

Conclusion: Reporting of test-treatment interventions is very poor, inadequate for understanding the results of these trials, and for comparing or translating results into clinical practice. Reporting needs to improve, with greater emphasis on describing the decision-making components of care pathways in both pragmatic and explanatory trials. Please see the companion paper to this article: http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0287-z .

Keywords: Diagnostic accuracy; Intervention reporting; Patient outcomes; RCT; Reporting quality; Test Evaluation; Test-treatment.

Fig. 1

The four steps of a test-treatment strategy using abdominal CT to diagnose appendicitis in patients presenting with right lower quadrant abdominal pain. The trial compared two diagnostic strategies for confirming or ruling out suspected acute appendicitis in adults presenting to the emergency department with right lower quadrant abdominal pain [18]. The routine approach of scanning all such patients with CT was compared with a new strategy of ordering CT only when indicated by specific signs and symptoms. In both arms the 4 test-treatment steps consist of testing (routine laboratory tests +/− a CT scan), a diagnostic decision (appendicitis, other disease, no disease), a management plan for each group (surgery or antibiotics to treat appendicitis, discharge for disease free patients, and treatment as necessary for alternative conditions), and delivery of these treatments. The trial’s primary outcome measured the proportion of removed appendices that were disease-free

Fig. 2

Identification of test-treatment RCTs from CENTRAL, searched 2009, Issue 2

Fig. 3

Proportion of trial arms describing each element of the test-treatment protocol. Some strategies did not require description of a test or treatment (or associated decision-making), for example when evaluating the addition of a new test to no test, or the removal of treatment. Denominators differed when test-treat strategies did not involve a test or a treatment (e.g., trials comparing the benefits of treating all vs. testing to select those for treatment [45]). The denominators for frequency calculations were reduced accordingly: Diagnostic method: Experimental n = 115, Control n = 92. Diagnostic decision: Experimental n = 115, Control n = 92. Treatment decision: Experimental n = 115, Control n = 95. Treatment method: Experimental n = 117, Control n = 102

Fig. 4

Example of a decision-tree graphic summarizing the 4 key components of one test-treatment intervention. Developed to illustrate an intervention evaluated in a published RCT [46]