A Survey of Vascular Surgery Program Directors: Perspectives following USMLE Step 1 Conversion to Pass/Fail and Virtual Only Interviews

Abstract

Background: Step 1 of the United States Medical Licensing Examination (USMLE), a historically important factor in the selection of vascular surgery residents and fellows, transitioned to completely pass/fail on January 26, 2022. Due to the COVID-19 pandemic, residency and fellowship interviews were conducted virtually during the 2020-21 and 2021-22 application cycles. Given these significant changes in the evaluation of candidates for residency training, we sought to understand vascular surgery program directors' (PDs) perspectives regarding the change in step 1 scoring and use of virtual interviews as well as determine which factors will assume importance when applying to vascular surgery training programs in the future.

Methods: A 26-item survey questionnaire was created using Qualtrics survey tools with questions regarding attitudes toward the change in step 1 scoring and virtual interviews, the importance of additional factors utilized by programs in selecting candidates for interviews and ranking applicants for residency/fellowship selection, and programmatic demographics. This was distributed anonymously to all vascular surgery training programs over a 2-week period using a comprehensive list of 249 unique email addresses created by cross-referencing the Accreditation Council for Graduate Medical Education list of certified vascular training programs with email addresses from individual program websites identifying current program leadership. Responses were analyzed with descriptive statistics with values listed as average Likert scale weight ± standard deviation (SD) or percentages.

Results: Sixty-eight of 249 (27.3%) program and associate PDs responded to the survey. Of which, 33.9% of respondents strongly disagreed with step 1 going completely pass/fail. In the absence of a scored USMLE step 1, letters of recommendation (average Likert scale weight ± SD, 4.43 ± 0.92), dedication to specialty (4.14 ± 1.03), and USMLE step 2 CK (4.06 ± 0.92) had the highest average scores for deciding which applicants to interview for integrated vascular surgery residency. For determining which candidates to interview for vascular surgery fellowship, letters of recommendation (4.51 ± 0.84), dedication to specialty (4.12 ± 0.90), and research (4.10 ± 0.80) had the highest average scores. For ranking residency candidates, the interviewee's perceived "fit" (4.61 ± 0.55), letters of recommendation (4.53 ± 0.76), and an overall interview experience (4.47 ± 0.62) had the highest average scores. Similarly, the factors with the highest average Likert scores for ranking fellowship candidates included the interviewee's perceived "fit" (4.69 ± 0.51), letters of recommendation (4.65 ± 0.52), and an overall interview experience (4.51 ± 0.59). The majority (72.2%) of PDs preferred in-person interviews; however, 50% of respondents were at least "somewhat satisfied" with virtual interviews during the 2021-22 application cycle as they could judge applicants' interview skills at least "moderately well." The minority (18.8%) who preferred virtual interviews most commonly noted a "reduction of the financial burden for applicants" as the reason for this preference.

Conclusions: Most vascular surgery program and associate PDs were dissatisfied with USMLE step 1 going pass/fail with most indicating prescreening applicants using both step 1 and step 2 clinical knowledge (CK) during the residency and fellowship selection processes. In the absence of a scored step 1, the top factors for interviewing and ranking integrated vascular surgery residency and fellowship candidates included letters of recommendation, dedication to specialty, research, USMLE step 2 CK, the interviewee's perceived "fit," and overall interview experience. Though most PDs preferred face-to-face interviews, they were overall at least "somewhat satisfied" with the virtual format that took place during the 2021-22 cycle.