Summative Supervisor Reporting: A Quality Performance Perspective

Abstract

Objective: This study aimed to quality assure Assigned Educational Supervisor (AES) reports, using UK Joint Committee on Surgical Training objective criteria, to evaluate contribution to Annual Review of Competence Progression.

Design: Consecutive 145 AES reports from 75 trainers regarding 68 Core Surgical Trainees were assessed from 9 hospitals (2 Tertiary centers [77 reports], 7 District General Hospitals [68 reports]). Reports were assessed by independent assessors based on free text related to performance mapped to curricular objectives, operative logbooks, and Clinical Supervisor reports, and overall summary grades assigned ranging from development required, adequate, good to excellent.

Setting: A core surgical training program serving a single UK (Wales) deanery.

Participants: Sixty-eight consecutively appointed core surgical trainees and 75 consultant surgeon trainers.

Results: Summary grades of adequate or above were achieved in 101 of 145 (69.7%) reports. Trainees' objective setting meetings were completed within 6 weeks of starting placements in 124 of 145 (85.5%). The proportions of AES reports containing free text commentary on curricular objectives, portfolio objectives, and operative logbook development were 128 of 145, 123 of 145, and 55 of 145, respectively. AES report quality was not associated with hospital status, subspecialty, or trainee grade. Female trainers were significantly more likely to provide reports graded as Good or Excellent compared with their male colleagues (7 of 12 vs. 27 of 133, χ² (2) = 9.389, p = 0.009). AES reports for male trainees were significantly more likely to be rated as further development required (40 of 85, 47.1%) when compared with female trainees (4 of 32, 12.5%, p = 0.007).

Conclusions: Three in ten AES reports were insufficient to contribute to objective Annual Review of Competence Progression outcomes and a gender gap was apparent related to engagement. AES trainers should provide more focus if this summative tool is to be an effective career progression metric.

Keywords: Core surgical training; Medical Knowledge; Patient Care; Practice-Based Learning and Improvement; Professionalism; Surgical training; Trainer quality.