Surgical implementation gap: an interrupted time series analysis with interviews examining the impact of surgical trials on surgical practice in England

Abstract

Objectives: Landmark studies published near the turn of the 21st century found an implementation gap concerning the effect of evidenced-based findings on clinical practice. The current study examines the uptake of six trials that produced actionable findings to describe the effects of evidence on practice and the reasons for those effects.

Design: A sequential, explanatory mixed methods study was conducted. First, a quantitative study assessed whether actionable findings from large, publicly funded elective surgical trials influenced practice. Subsequently, qualitative interviews were conducted to explain the quantitative findings.

Setting: Changes in NHS-funded practice were tracked across hospitals in England. Interviews were conducted online.

Data and participants: The six surgical trials were funded and published by England's National Institute for Health Research's Health Technology Assessment programme between 2006 and 2015. Quantitative time series analyses used data about the frequencies or proportions of relevant surgical procedures conducted in England between 2001 and 2020. Subsequently, qualitative interviews were conducted with 25 participants including study authors, surgeons and other healthcare staff in the supply chain. Transcripts were coded to identify major temporal events and Consolidated Framework for Implementation Research (CFIR) domains/constructs that could influence implementation. Findings were synthesised by clinical area.

Results: The quantitative analyses reveal that practice changed in accordance with findings for three trials. In one trial (percutaneous vs nasogastric tube feed after stroke), the change took a decade to occur. In another (patella resurfacing), change anticipated the trial findings. In the third (abdominal aortic aneurysm repair), changes tracked the evolving evidence base. In the remaining trials (two about varicose veins and one about gastric reflux), practice did not change in line with findings. For varicose veins, the results were superseded by a further trial. For gastric reflux, surgical referrals declined as medical treatment increased. The exploratory qualitative analysis informed by CFIR found that evidence from sources apart from the trial in question was mentioned as a reason for non-adoption in the three trials where evidence did not affect practice and in the trial where uptake was delayed. There were no other consistent patterns in the qualitative data.

Conclusion: While practice does not always change in the direction indicated by clinical trials, our results suggest that individuals, official committees and professional societies do assimilate trial evidence. Decision-makers seem to respond to the totality of evidence such that there are often plausible reasons for not adopting the evidence of any one trial in isolation.

Keywords: Evidence-based medicine; Implementation science; Performance measures; Surgery.

Figure 1

FOOD trial timeline. Note: This figure shows the percentage of admissions with primary diagnosis of stroke where the patient stayed overnight and had a PEG recorded during their stay, along with events potentially influencing implementation of evidence-based findings from the FOOD trial. FOOD, Feed Or Ordinary Diet; HTA, Health Technology Assessment; NG, nasogastric; PEG, percutaneous endoscopic gastrostomy.

Figure 2

REFLUX trial timeline. Note: This figure shows the number of admissions with a primary diagnosis of gastro-oesophageal reflux disease where the patient underwent anti-reflux fundoplication surgery, along with events potentially influencing implementation of evidence-based findings from the REFLUX trials. HTA, Health Technology Assessment; NICE, National Institute for Health and Care Excellence; REFLUX, Randomised Evaluation oF Laparoscopic sUrgery for refluX.

Figure 3

EVAR trial timeline. Note: This figure shows the percentage of EVAR and open surgeries performed for elective admissions with a primary diagnosis, along with events potentially influencing implementation of evidence-based findings from the EVAR trial. EVAR, EndoVascular Aneurysm Repair; HTA, Health Technology Assessment.

Figure 4

KAT trial timeline. Note: This figure shows the percentage of patients having total knee replacement, who also have a code for resurfacing the patella, along with events potentially influencing implementation of evidence-based findings from the KAT trials. The blue line shows data reported in the Hospital Episodes Statistic database and the purple line shows data reported in the National Joint Registry (NJR). Data from the NJR are only published annually, so the true quarterly line may not be as smooth as is shown. KAT, Knee Arthroplasty Trial; NICE, National Institute for Health and Care Excellence.

Figure 5

REACTIVE and CLaSS trials timelines. Note: This figure shows the percentage of varicose vein procedures undertaken in hospital (either inpatient or outpatient settings) by the type of procedure, along with events potentially influencing implementation of evidence-based findings from the REACTIV and CLaSS trials. CLaSS, Comparison of LAser, Surgery and foam Sclerotherapy; HTA, Health Technology Assessment; REACTIV, Randomised and Economic Assessment of Conservative and Therapeutic Interventions for Varicose Veins.