Serial image interpretation tasks improve accuracy and increase confidence in Level 1 echocardiography reporting: a pilot study

Abstract

Background: Focused echocardiography is increasingly used in acute and emergency care, with point-of-care ultrasound integrated into several specialist training curricula (e.g. Emergency Medicine, Cardiology, Critical Care). Multiple accreditation pathways support development of this skill but there is scant empirical evidence to inform selection of teaching methods, accreditation requirements or quality assurance of education in focussed echocardiography. It has also been noted that access to in-person teaching can be a barrier to completing accreditation programmes, and that this may affect learners disproportionately depending on the location or nature of their institution. The purpose of the study was to determine whether serial image interpretation tasks as a distinct learning tool improved novice echocardiographers' ability to accurately identify potentially life-threatening pathology from focused scans. We also aimed to describe the relationship between accuracy of reporting and participants' confidence in those reports, and to assess users' satisfaction with a learning pathway that could potentially be delivered remotely.

Methods: 27 participants from a variety of healthcare roles completed a program of remote lectures and 2 in-person study days. During the program they undertook 4 'packets' of 10 focused echocardiography reporting tasks (total = 40) based on images from a standardised dataset. Participants were randomized to view the scans in varying orders. Reporting accuracy was compared with consensus reports from a panel of expert echocardiographers, and participants self-reported confidence in their image interpretation and their satisfaction with the learning experience.

Results: There was a stepwise improvement in reporting accuracy with each set of images reported, from an average reporting score of 66% for the 1st packet to 78% for the 4th packet. Participants felt more confident in identifying common life-threatening pathologies as they reported more echocardiograms. The correlation between report accuracy and confidence in the report was weak and did not increase during the study (r_s = 0.394 for the 1st packet, r_s = 0.321 for the 4th packet). Attrition during the study related primarily to logistical issues. There were high levels of satisfaction amongst participants, with most reporting that they would use and / or recommend a similar teaching package to colleagues.

Conclusions: Healthcare professionals undertaking remote training with recorded lectures, followed by multiple reporting tasks were capable of interpreting focused echocardiograms. Reporting accuracy and confidence in identifying life-threatening pathology increased with the number of scans interpreted. The correlation between accuracy and confidence for any given report was weak (and this relationship should be explored further given the potential safety considerations). All components of this package could be delivered via distance learning to enhance the flexibility of echocardiography education.

Keywords: Distance education; Echocardiography; Focused echocardiography; Image interpretation; Medical education; Pocus; Point-of-care; Skill acquisition; Ultrasound.

Fig. 1

Primary outcome: Change in Participant Packet Difference (PPD) from baseline

Fig. 2

Subjective confidence (on a scale of 0–100)

Fig. 3

Correlation between report accuracy and subjective confidence in report. A first packet attempted (timepoint 1); n = 266*; mean reporting accuracy 66%; mean confidence in report 24%; line of best fit y = 0.4117x-3.0899; r_s = 0.394; p-value < 0.00001. B second packet attempted (timepoint 2); n = 259*; mean reporting accuracy 73%; mean confidence in report 31%; line of best fit y = 0.4558x-2.4703; r_s = 0.339; p-value < 0.00001. C third packet attempted (timepoint 5); n = 268*; mean reporting accuracy 77%; mean confidence in report 43%; line of best fit y = 0.5918x-2.1069; r_s = 0.392; p-value < 0.00001. D fourth packet attempted (timepoint 6); n = 266*; mean reporting accuracy 78%; mean confidence in report 46%; line of best fit y = 0.5105x + 6.0985; r_s = 0.321; p-value < 0.00001. *1080 echocardiograms reported, but subjective confidence only rated for 1059 echocardiograms (98.1%)