A Comparison of Two Debriefing Rubrics to Assess Facilitator Adherence to the PEARLS Debriefing Framework

Abstract

Introduction: Many educators have adopted the Promoting Excellence and Reflective Learning in Simulation (PEARLS) model to guide debriefing sessions in simulation-based learning. The PEARLS Debriefing Checklist (PDC), a 28-item instrument, and the PEARLS Debriefing Adherence Rubric (PDAR), a 13-item instrument, assess facilitator adherence to the model. The aims of this study were to collect evidence of concurrent validity and to evaluate their unique strengths.

Methods: A review of 130 video recorded debriefings from a synchronous high-fidelity mannequin simulation event involving third-year medical students was undertaken. Each debriefing was scored utilizing both instruments. Internal consistency was determined by calculating a Cronbach's α. A Pearson correlation was used to evaluate concurrent validity. Discrimination indices were also calculated.

Results: Cronbach's α values were 0.515 and 0.714 for the PDAR and PDC, respectively, with ≥0.70 to ≤0.90 considered to be an acceptable range. The Pearson correlation coefficient for the total sum of the scores of both instruments was 0.648, with a values between ±0.60 and ±0.80 considered strong correlations. All items on the PDAR had positive discrimination indices; 3 items on the PDC had indices ≤0, with values between -0.2 and 0.2 considered unsatisfactory. Four items on both instruments had indices >0.4, indicating only fair discrimination between high and low performers.

Conclusions: Both instruments exhibit unique strengths and limitations. The PDC demonstrated greater internal consistency, likely secondary to having more items, with the tradeoff of redundant items and laborious implementation. Both had concurrent validity in nearly all subdomains. The PDAR had proportionally more items with high discrimination and no items with indices ≤0. A revised instrument incorporating PDC items with high reliability and validity and removing those identified as redundant or poor discriminators, the PDAR 2, is proposed.