A pilot study of orthopaedic resident self-assessment using a milestones' survey just prior to milestones implementation

Abstract

Objective: To pilot test if Orthopaedic Surgery residents could self-assess their performance using newly created milestones, as defined by the Accreditation Council on Graduate Medical Education.

Methods: In June 2012, an email was sent to Program Directors and administrative coordinators of the 154 accredited Orthopaedic Surgery Programs, asking them to send their residents a link to an online survey. The survey was adapted from the Orthopaedic Surgery Milestone Project. Completed surveys were aggregated in an anonymous, confidential database. SAS 9.3 was used to perform the analyses.

Results: Responses from 71 residents were analyzed. First and second year residents indicated through self-assessment that they had substantially achieved Level 1 and Level 2 milestones. Third year residents reported they had substantially achieved 30/41, and fourth year residents, all Level 3 milestones. Fifth year, graduating residents, reported they had substantially achieved 17 Level 4 milestones, and were extremely close on another 15. No milestone was rated at Level 5, the maximum possible. Earlier in training, Patient Care and Medical Knowledge milestones were rated lower than the milestones reflecting the other four competencies of Practice Based Learning and Improvement, Systems Based Practice, Professionalism, and Interpersonal Communication. The gap was closed by the fourth year.

Conclusions: Residents were able to successfully self-assess using the 41 Orthopaedic Surgery milestones. Respondents' rate improved proficiency over time. Graduating residents report they have substantially, or close to substantially, achieved all Level 4 milestones. Milestone self-assessment may be a useful tool as one component of a program's overall performance assessment strategy.

Keywords: competency based education; graduate medical education; milestones; postgraduate medical education; self-assessment.

Figure 1

Trajectory of aggregate milestone acquisition combined for competencies of Patient Care (PC), Medical Knowledge (MK), and pooled Systems Based Practice (SBP)-Practice Based Learning and Improvement (PBLI)-Professionalism (P)-Interpersonal and Communication (IC) by postgraduate years 1-5

Figure 2

Trajectory of milestone acquisition by postgraduate year for each of the general competency domains of Systems Based Practice (SBP), Practice Based Learning and Improvement (PBLI), Professionalism (P), and Inter-personal and Communication Skills (IC) by postgraduate years 1-5. *st: systems thinking (SBP); interprof: Interprofessional teams, patient safety and quality (SBP); tech: technology (SBP); sdl: self-directed learning (PBLI); laa: locate, appraise and assimilate evidence (PBLI); account: accountability to patients, society and the profession (P); comm: communication (IC); team: teamwork (IC)