HELP! Problems in executing a pragmatic, randomized, stepped wedge trial on the Hospital Elder Life Program to prevent delirium in older patients

Abstract

Background: A pragmatic, stepped wedge trial design can be an appealing design to evaluate complex interventions in real-life settings. However, there are certain pitfalls that need to be considered. This paper reports on the experiences and lessons learned from the conduct of a cluster randomized, stepped wedge trial evaluating the effect of the Hospital Elder Life Program (HELP) in a Dutch hospital setting to prevent older patients from developing delirium.

Methods: We evaluated our trial which was conducted in eight departments in two hospitals in hospitalized patients aged 70 years or older who were at risk for delirium by reflecting on the assumptions that we had and on what we intended to accomplish when we started, as compared to what we actually realized in the different phases of our study. Lessons learned on the design, the timeline, the enrollment of eligible patients and the use of routinely collected data are provided accompanied by recommendations to address challenges.

Results: The start of the trial was delayed which caused subsequent time schedule problems. The requirement for individual informed consent for a quality improvement project made the inclusion more prone to selection bias. Most units experienced major difficulties in including patients, leading to excluding two of the eight units from participation. This resulted in failing to include a similar number of patients in the control condition versus the intervention condition. Data on outcomes routinely collected in the electronic patient records were not accessible during the study, and appeared to be often missing during analyses.

Conclusions: The stepped wedge, cluster randomized trial poses specific risks in the design and execution of research in real-life settings of which researchers should be aware to prevent negative consequences impacting the validity of their results. Valid conclusions on the effectiveness of the HELP in the Dutch hospital setting are hampered by the limited quantity and quality of routine clinical data in our pragmatic trial. Executing a stepped wedge design in a daily practice setting using routinely collected data requires specific attention to ethical review, flexibility, a spacious time schedule, the availability of substantial capacity in the research team and early checks on the data availability and quality.

Trial registration: Netherlands Trial Register, identifier: NTR3842 . Registered on 24 January 2013.

Keywords: Delirium; Pragmatic trial; Stepped wedge trial.

Fig. 1

Schematic representation of the planned stepped wedge, cluster randomized trial study design. Orange boxes, cluster not exposed to intervention. Green boxes, cluster exposed to intervention. * Participating units were to be randomized to clusters 1–8 and would all start with a control period. Every 3 months, two units would cross over to use the intervention

Fig. 2

Flow chart of the patients approached for participation and the study population included pre-Hospital Elder Life Program (pre-HELP) and during the HELP intervention and follow-up. * It is unknown whether patients were approached in the pre-HELP or in the HELP period

Fig. 3

Schematic representation of the inclusion of patients per period in each cluster in the actual study design of the stepped wedge, cluster randomized trial. Orange boxes, cluster not exposed to intervention. Green boxes, cluster exposed to intervention. Beige boxes, cluster in which no patients were included in the specified period. * Participating units 1–10 were randomized to clusters subsequently starting to use the intervention

Fig. 4

Estimates of the incidence of delirium in each cluster during the pre-Hospital Elder Life Program (pre-HELP) (left panel) and the HELP (right panel) periods