Standardized Letter of Recommendation: Can Everyone Be Awesome?

Abstract

Objective: Standardized letters of recommendation (SLOR) are hypothesized to decrease bias and provide consistent domains for evaluation. However, their ability to differentiate among applicants is unknown. The utilization and functionality of SLOR and the impact of SLOR domain rating on matching for colon and rectal surgery (CRS) residency applicants have yet to be assessed.

Design: Descriptive statistics and bivariate analysis were employed. Applicants were categorized into 3 groups; Top-tier(TT): applicants rated 100% Excellent/Very Good; Mid-Tier(MT) applicants rated 80-99% Excellent/Very Good; and Non-Top Tier(NTT) applicants rated <80% Excellent/Very Good.

Setting: University of Hospitals Cleveland Medical Center.

Participants: SLORs submitted to a single colorectal surgery residency in 2019 were analyzed RESULTS: A total of 101 applicants were included, 54 (53.5%) of the applicants were male. 75 (74.2%) applicants who applied to our residency matched into a CRS residency, compared to the national rate of 66%. Of the 101 applicants with SLOR, 54 (53.5%) were categorized as TT, 26 (25.7%) as MT, and 21(20.8%) as NTT. The univariable analysis demonstrated a statistically significant difference in research experience (p=0.029) and match status (p=0.01) between applicant tiers. There were no statistically significant differences between applicant-tier and demographics, foreign medical graduates (FMG), H-indices, ABSITE scores, type of residency, preliminary year, completing an unaccredited CRS, and applicants with an additional degree. On multivariable analysis age (OR=0.65; CI=0.48-0.87) and FMG applicants (OR=0.05; CI=0.01-0.44) were inversely associated with successfully matching. Compared to TT applicants, MT (OR=0.07; CI=0.01-0.57) and NTT (OR=0.04; CI=0.01-0.34) applicants were inversely associated with a successful match. Individuals who completed research prior to residency but after medical school was associated with successfully matching (p=0.009).

Conclusions: The presence of MT and NTT ratings is associated with failure to match and may represent an area of concern for CRS programs rather than a tool to discern differences between candidates.

Keywords: Association of Program Directors for Colon and Rectal Surgery; Colon and Rectal Residency; Letters of Recommendation; Matching; Standardized letter of recommendation.