Analyses of Orthopaedic Surgery Residency Interviews

Abstract

Interviews are a critical component of orthopaedic surgery residency selection for both the applicant and the program. Some institutions no longer report Alpha Omega Alpha (AOA) designation or class rank, and US Medical Licensing Examination (USMLE) Step 1 recently switched to pass/fail scoring. During the coronavirus disease 2019 (COVID-19) pandemic, all Accreditation Council for Graduate Medical Education programs conducted virtual interviews and subinternship rotations were restricted. These changes offer significant challenges to the residency match process. The purpose of this study was to examine the residency applicant interview and ranking process at a large urban academic university setting. We hypothesized that large variability exists among evaluations submitted by faculty interviewers and also that applicant academic factors (i.e., USMLE Step 1 score) would show association with final ranking.

Methods: We retrospectively reviewed the 2020-2021 and 2021-2022 residency interview cycles, both conducted virtually due to the COVID-19 pandemic. Residency application (i.e., applicant demographic and academic backgrounds) and interview data (i.e., faculty interviewer scores) were recorded. Interobserver reliability among faculty interviewers was calculated. Statistical analysis was performed to determine factors associated with ranking of applicants.

Results: There were 195 included applicants from the 2020 and 2021 interview cycles. There was no true agreement of interviewers' scoring of shared applicants (kappa intraclass coefficient range 0-0.2). Applicant factors associated with being ranked include applying to the match for the first time, USMLE Step 1 and 2 scores, educational break (vs. consecutive completion of college and medical school in 4 years each), higher class rank, and greater interviewer scores. Factors associated with better rank included additional degrees (i.e., PhD or MBA), couples match, AOA designation, educational break, underrepresented minority status, and notable attributes (i.e., collegiate athletics or Eagle Scout participation). Factors associated with worse rank included male sex, international medical graduate, prior match history, science major, extended research (i.e., >1 year spent in a research role), and home medical school students.

Conclusions: There was significant variability and no reliability at our institution among faculty interviewers' applicant ratings. Being ranked was based more on academic record and interview performance while final rank number seemed based on applicant qualities. The removal of merit-based objective applicant measurements offers challenges to optimal residency applicant and program match.

Level of evidence: III (retrospective cohort study).

Fig. 1

Diagram summarizing the annual orthopaedic resident interview and scoring process at our institution. PD = program director.