Electronic checklists improve referral letters in gastroenterology: a randomized vignette survey

Abstract

Objective: Investigate whether gastroenterologists rate the quality of referral letters higher if electronic dynamic checklist items are added to a standard free-text referral letter. Assess how this affects the gastroenterologists' assessment of the patient's need for healthcare and the agreement between their assessments.

Design: Randomized vignette study.

Setting: Norwegian primary gastroenterology services.

Participants: Thirty-two Norwegian gastroenterologists.

Intervention: Between June 2015 and January 2016, participants were recruited through an open invitation to all members of the Norwegian Society of Gastroenterology. They were asked to rate 16 referral letters (vignettes) in a web interface: eight letters in free text following a general template and eight letters based on a general referral template combined with diagnosis-specific checklist items. The study was completed in two subsequent rounds ≥3 months apart.

Main outcome measures: Quality of referral letters assessed on a rating scale from 0 to 10. Agreement in the referral assessment and accuracy of the selection of correct preliminary diagnosis and appropriate work-up.

Results: The mean quality assesses on the rating scale was 7.0 (95% confidence interval [CI] 6.8-7.2) for all letters combined (n = 511), 6.5(CI 6.2-6.8) for the free-text referrals (n = 256) and 7.5(CI 7.3-7.7) for the checklist referrals (n = 255) (P < 0.001, paired t-test). No difference was observed in the triage of the patients, but fewer gastroenterologists felt the need to collect additional information about the patients in the checklist group.

Conclusion: Checklist items may ease the assessment of the referrals for gastroenterologists. We were not able to show that checklists significantly influence the management of patients.

Figure 1

Flowchart.

Figure 2

Gastroenterologists’ rating of the quality of the referral letters with and without checklist on a 0–10 rating scale, stratified by indication. Footnote: P-value is calculated using paired t-test.

Figure 3

Gastroenterologists’ assessment of waiting time, with and without checklists, stratified by indication.