Predicting Academic Career Placement via Development of Novel Intra-residency Metrics

Abstract

Objective: Increasing centralization of high-level neurosurgical practice at academic centers has increased the need for academic neurosurgeons. The lack of systematic metrics-based analyses among neurosurgery trainees and the recent pass/fail U.S. Medical Licensing Examination system necessitates a multiparametric approach to assess academic success among trainees.

Methods: We conducted a comprehensive analysis of the University of Miami residency program using 2 data sets, one containing applicants' pre-residency metrics and a second containing trainees' intra-residency metrics. Intra-residency metrics were subjectively and anonymously assessed by faculty. Univariate and multivariate logistic regression analyses were performed to determine differences among academic and non-academic neurosurgeons and identify predictors of academic careers.

Results: Academic neurosurgeons had a significantly higher median Step 1 percentile relative to non-academic neurosurgeons (P = 0.015), and medical school ranking had no significant impact on career (P > 0.05). Among intra-residency metrics, academic neurosurgeons demonstrated higher mean rating of leadership skills (mean difference [MD] 0.46, P = 0.0011), technical skill (MD 0.42, P = 0.006), and other intra-residency metrics. Higher administrative and leadership skills were significantly associated with increased likelihood of pursuing an academic career (odds ratio [OR] 9.03, 95% CI [2.296 to 49.88], P = 0.0044). Clinical judgment and clinical knowledge were strongly associated with pursuit of an academic career (OR 9.33 and OR 9.32, respectively, with P = 0.0060 and P = 0.0010, respectively).

Conclusions: Pre-residency metrics had little predictive value in determining academic careers. Furthermore, medical school ranking does not play a significant role in determining a career in academic neurosurgery. Intra-residency judgment appears to play a significant role in career placement, as academic neurosurgeons were rated consistently higher than their non-academic peers in multiple key parameters by their attending physicians.

Keywords: Academic neurosurgery; Neurosurgery residency; USMLE.

Figure 1.

Differences in pre-residency metrics among academic versus non-academic neurosurgeons in cohort 1. With respect to medical school ranking (A), academic neurosurgeons had a higher mean medical school ranking relative to non-academic neurosurgeons, though this difference was not significant (36.85 ± 2.99 vs. 40.69 ± 2.89, P = 0.36). Academic neurosurgeons also had a significantly higher median Step 1 percentile (B) relative to non-academic neurosurgeons (73.56 IQR 26 vs. 66.67 IQR 28, P = 0.015). Similarly, there were no significant differences in USMLE Step 2 scores (P = 0.11) (C), or in number of pre-residency publications (P = 0. 61) (D).

Figure 2.

Correlation matrix of various applicant metrics in cohort one across all surgeons (A) and those in academia (B). Across all surgeons, there was a significant association between number of pre-residency publications and medical school ranking (R = 0.36, P < 0.0001). There was also a significant correlation between number of pre-residency publications and USMLE Step 1 score (R = −0.21, P = 0.009). A similar trend was observed among academic neurosurgeons, with significant correlations between number of pre-residency publications and medical school ranking (R = 0.50, P = 0. 000002) and between pre-residency publications and USMLE Step 1 score (R = −0.32, P = 0.01). There were no significant correlations between pre-residency metrics among non-academic neurosurgeons (P > 0.05 for all, C).

Figure 3.

Differences in intra-residency metrics among academic versus non-academic neurosurgeons in cohort 2. Academic neurosurgeons demonstrated higher (A) mean rating of leadership skills (MD 0.46, P = 0.0011), (B) mean rating of technical skill (MD 0.42, P = 0.0060), (C) mean rating of research skill (MD 0.53, P = 0.0017), (D) rating of clinical knowledge (MD= 0. 36, P = 0.0037), (E) rating of clinical judgment (MD=0. 43, P = 0.0015), and (F) rating of patient relationships (MD 0.31, P = 0.01). There were no significant differences in (G) rating of colleague relationships between academic and non-academic neurosurgeons (MD 0.24, P = 0.09), or in (H) number of intra-residency publications (P = 0.26).

Figure 4.

Correlation matrix of various intra-residency metrics in cohort 1 across all surgeons (A) and those in academia (B). Across all surgeons, there were no significant associations between intra-residency metrics and number of publications (P > 0.05 for all), while all intra-residency metrics were significantly correlated with one another (P < 0.00001) across all metrics. These trends were preserved among academic neurosurgeons.