A think-aloud study to inform the design of radiograph interpretation practice

Abstract

Models for diagnostic reasoning in radiology have been based on the observed behaviors of experienced radiologists but have not directly focused on the thought processes of novices as they improve their accuracy of image interpretation. By collecting think-aloud verbal reports, the current study was designed to investigate differences in specific thought processes between medical students (novices) as they learn and radiologists (experts), so that we can better design future instructional environments. Seven medical students and four physicians with radiology training were asked to interpret and diagnose pediatric elbow radiographs where fracture is suspected. After reporting their diagnosis of a case, they were given immediate feedback. Participants were asked to verbalize their thoughts while completing the diagnosis and while they reflected on the provided feedback. The protocol analysis of their verbalizations showed that participants used some combination of four processes to interpret the case: gestalt interpretation, purposeful search, rule application, and reasoning from a prior case. All types of processes except reasoning from a prior case were applied significantly more frequently by experts. Further, gestalt interpretation was used with higher frequency in abnormal cases while purposeful search was used more often for normal cases. Our assessment of processes could help guide the design of instructional environments with well-curated image banks and analytics to facilitate the novice's journey to expertise in image interpretation.

Keywords: Cognition; Education (medical); Emergency medicine; Instructional design; Radiology.

Fig. 1

Typical pediatric elbow radiograph. The radiologic anatomy of a pediatric elbow showing some of the features that would be considered in searching for an abnormality such as a fracture

Fig. 2

A screen capture from the ImageSim learning system used in the study. Shown is the feedback page demonstrating: a the learner’s assignation of where the fracture lies (red marker) which would have been placed by the learner on a prior screen, on an otherwise unmarked radiograph; b three forms of feedback including overall correctness (green checkmark), text of radiologist report, and yellow target area pre-assigned by an expert radiologist. (Color figure online)

Fig. 3

Mixed method analysis of verbal protocol and log file data

Fig. 4

Certainty by expertise level. As expected, experts were found to be more certain in their diagnoses, especially for abnormal cases. Plot shows predicted margins for a logistic regression model predicting use of the qualifier “Definitely” based on case type (Normal, Abnormal) and level of expertise. Whisker shows upper bound of 95% CI