A qualitative study exploring the acceptability of the McNulty-Zelen design for randomised controlled trials evaluating educational interventions

Abstract

Background: Traditional randomised controlled trials evaluating the effect of educational interventions in general practice may produce biased results as participants know they are being evaluated. We aimed to explore the acceptability of a McNulty-Zelen Cluster Randomised Control Trial (CRT) design which conceals from educational participants that they are in a RCT. Consent is obtained from a trusted third party considered appropriate to give consent on participants' behalf, intervention practice staff then choose whether to attend the offered education as would occur with normal continuing professional development.

Methods: We undertook semi structured telephone interviews in England with 16 general practice (GP) staff involved in a RCT evaluating an educational intervention aimed at increasing chlamydia screening tests in general practice using the McNulty-Zelen design, 4 Primary Care (PC) Research Network officers, 5 Primary Care Trust leads in Public or sexual health, and one Research Ethics committee Chair. Interviews were undertaken by members of the original intervention evaluation McNulty-Zelen design RCT study team. These experienced qualitative interviewers used an agreed semi-structured interview schedule and were careful not to lead the participants. To further mitigate against bias, the data analysis was undertaken by a researcher (CR) not involved in the original RCT.

Results: We reached data saturation and found five main themes; Support for the design: All found the McNulty-Zelen design acceptable because they considered that it generated more reliable evidence of the value of new educational interventions in real life GP settings. Lack of familiarity with study design: The design was novel to all. GP staff likened the evaluation using the McNulty-Zelen design to audit of their activities with feedback, which were to them a daily experience and therefore acceptable. Ethical considerations: Research stakeholders considered the consent procedure should be very clear and that these trial designs should go through at least a proportionate ethical review. GP staff were happy for the PCT leads to give consent on their behalf. GP research capacity and trial participation: GP staff considered the design increased generalisability, as staff who would not normally volunteer to participate in research due to perceived time constraints and paperwork might do so. Design 'worth it': All interviewees agreed that the advantages of the "more accurate" or "truer" results and information gained about uptake of workshops within Primary Care Trusts (PCTs) outweighed any disadvantages of the consent procedure.

Discussion: Our RCT was evaluating the effect of an educational intervention to increase chlamydia screening tests in general practices where there was routine monitoring of testing rates; our participants may have been less enthusiastic about the design if it had been evaluating a more controversial educational area, or if data monitoring was not routine.

Implications: The McNulty-Zelen design should be considered for the evaluation of educational interventions, but these designs should have clear consent protocols and proportionate ethical review.

Trial registration: The trial was registered on the UK Clinical Research Network Study Portfolio database. UKCRN9722 .

Fig. 1

Randomisation and consent with McNulty-Zelen design and usual cluster RCT; detailed legend, comparative processes for traditional cluster RCT and the McNulty-Zelen design used in this study

Fig. 2

Recruitment of GP staff for interviews; detailed legend, Flowchart depicting process of recruitment of GP staff for interviews

Fig. 3

Thematic Map of Main Themes found in qualitative interviews; detailed legend, Map of 5 themes from Thematic Analysis of all interviews