Investigator-initiated versus industry-sponsored trials - visibility and relevance of randomized controlled trials in clinical practice guidelines (IMPACT)

Abstract

Background: The goal of evidence-based medicine is to make clinical decisions based on the best available, relevant evidence. For this to be possible, studies such as randomized controlled trials (RCTs), which are widely considered to provide the best evidence of all forms of primary research, must be visible and have an impact on clinical practice guidelines. We further investigated the impact of publicly and commercially sponsored RCTs on clinical practice guidelines by measuring direct and indirect impactful citations and the time to guideline impact.

Methods: We considered the sample from the IMPACT study, where a total of 691 RCTs (120 German investigator-initiated trials (IITs), 200 international IITs, 171 German industry-sponsored trials (ISTs) and 200 international ISTs) was sampled from registries (DFG-/BMBF-Websites, the German Clinical Trials Register, and from ClinicalTrials.gov) and followed prospectively. First, all eligible IITs were sampled. Then, ISTs were randomly selected while ensuring balance across certain trial characteristics. Next, the corresponding publications in the form of original research articles were identified. A search was then conducted for (1) systematic reviews (SRs) citing these articles and (2) clinical practice guidelines (CPGs) that cited either the original articles or the SRs. The methods and results of this effort were already published. In this investigation we aimed to better characterize the impact of RCTs in CPGs. Therefore, we identified all citations of the original articles and SRs in the citing CPGs and classified them into impactful and non-impactful. This allowed us to calculate an estimate for the guideline impact of a trial. In addition, we estimated the time-to-guideline-impact, defined as the time to a direct and indirect impactful citation in a CPG. Direct means that the publication of a trial was cited in the main text of a CPG. Indirect means that the publication was cited and included in the findings of a SR and the SR was cited in the main text of a CPG. We also investigated to what extent pre-defined study characteristics influenced the guideline impact using multivariable negative binomial regression as well as the time-to-guideline impact using multivariable Cox proportional hazards regression.

Results: Overall, 22% of RCTs impacted a CPG. For international ISTs, only 15% of trials had an impact in CPGs. Overall, of the 405 associated guidelines, 331 were impacted. Larger trials were associated with more impactful main text citations in CPGs and earlier time-to-guideline impact, while international industry-sponsored trials were associated with smaller impact on CPGs and longer time-to-guideline impact. IITs funded by governmental bodies in Germany reached an impact on CPGs that is on par with German ISTs or international IITs and ISTs.

Conclusion: This study demonstrated that a considerable number of trials previously identified as being linked to CPGs have had impact in those CPGs (85%). International ISTs seem to have a lower impact on CPGs, and fewer of them influence CPGs at all.

Keywords: Access to information; Clinical decision-making; Evidence-based medicine; Health impact assessment; Knowledge translation; Practice guidelines as topic; Publishing; Randomized controlled trials as topic; Registries.

Fig. 1

Pseudo-tabular data representation. Displayed are - from left to right: the presence or possible absence of entities like articles and SRs in between the possible connections between RCTs and CPGs in our data, the number of citations of the referenced entity in different parts of the CPG: Main Text (Main) or Appendix/Supplement (Supp) and whether a path was impactful. We displayed all possible paths from RCTs to CPGs (from top to bottom): Case 1: the CPG cited an article, but the article is excluded in the CPG (symbolized by dotted arrow), Case 2: the CPG cited an article and the article is included (symbolized by solid arrow), Case 3: the CPG cited but excluded or Case 4: included a SR and the SR cited an article but excluded it, Case 5: the CPG cited and included or Case 6. excluded an SR and the SR cites and included it, Case 7: the CPG cites the trial directly e.g., the trial registry number. In all these cases, the reference was not included in the CPG in our dataset, therefore we do not show the hypothetical case of a direct RCT citation with inclusion in CPG here. Inclusion in guideline was defined as having at least one citation in the meaningful sections of a guideline, thus ignoring the citations in the Appendix/Supplement (this only holds for cases 2, 4, and 5). We defined a path to CPG as impactful when in each step of citation the cited reference was included in the citing entity (SR or CPG)

Fig. 2

Stages of impact for one trial of the IMPACT study. From bottom to top, we displayed the information from the trial registry, then the publications as journal articles, the SRs citing these articles and the CPGs citing either the articles or the SRs. Dotted lines indicate exclusion in the citing entity, solid lines indicate inclusion. To be able to visually differentiate between CPGs citing SRs and articles these connections are color-coded with purple and blue respectively. Additionally, we displayed for the articles, whether they are a method article (only describing the methodology of a trial) or contained results

Fig. 3

Histograms for the outcomes impact-on-CPGs (left) and impacted-CPGs per trial (right). Displayed are the absolute values, which should be interpreted relative to the cohort sizes 120, 200, 171, and 200, respectively

Fig. 4

Flowchart data representation. Total number of trials, published articles, SR citing these articles and CPGs citing the articles or the SRs. Dotted arrows indicate non-inclusion in the citing entity, solid arrows indicate inclusion. Paths which were taken by a considerable (>100) number of trials are thicker. Since for example trials with articles can be cited with their articles in SRs and be excluded as well as included, the sum of 238 trials and 309 trials merely represents an upper bound for the total number of distinct trials being associated with SRs (360). The same type of argument applies to the other paths. The proportion of the cohorts is given between brackets (“[ ]”) in the usual manner

Fig. 5

Kaplan-Meier estimates for the probability of having a guideline impact are displayed as the cumulative event. The time-to-event was the time from trial start to the publication of the first CPG impacted by a trial. If a trial did not impact a CPG until the end of the observation period, it was treated as right censored with the censoring date 31st August 2019. An event happened for 153 of 691 studies. The risk table shows the trials still at risk at each year since study start stratified by cohort