A Computer Simulation Model to Analyze the Application Process for Competitive Residency Programs

Abstract

Background: The residency match process for competitive specialties hinders programs' ability to holistically review applications.

Objective: A computer simulation model of the residency application process was created to test the hypotheses that (1) it is advantageous to medical students to apply to the maximum number of programs under the current system, and (2) including a medical student's residency program preferences at the beginning of the application process improves the efficiency of the system for applicants and programs as quantified by the number of interview invitations received.

Methods: The study was conducted in 2016 using 2014 Otolaryngology Match data. A computer model was created to perform simulations for multiple scenarios to test the hypotheses. Students were assigned scores representing easy and hard metrics and program preferences, simulating a mixture of individual student preference and general program popularity.

Results: We modeled a system of 99 otolaryngology residency programs with 292 residency spots and 460 student applicants. While it was individually advantageous for an applicant to apply to the maximum number of programs, this led to a poor result for the majority of students when all applicants undertook the strategy. The number of interview invitations improved for most applicants when preference was revealed.

Conclusions: Offering applicants an option to provide program preference improves the practical number of interview invitations. This enables programs to review applicants holistically-instead of using single parameters such as United States Medical Licensing Examination scores-which facilitates a selection of applicants who will be successful in residency.

Figure 1

Interviews for Average Student by Number of Applications^{a a} Number of interviews received versus number of applications submitted for index student when all other students submitted a constant number of applications. The index student in this case had average easy and hard scores. The 3 curves show a scenario when all other applicants apply to a fixed number of programs (10, 20, and 30).

Figure 2

Interviews for Students with Varying Characteristics and Preference Sharing^{a a} Number of interview invitations for index student with varying easy and hard to review characteristics relative to the mean. The figure plots the interview invitations received based on the number of applications submitted and the provision of preference information. Each subplot represents an index student with hard and easy scores of either –2, –1, 0, +1, or +2 standard deviations from the mean. The x-axis indicates the number of applications sent out by all students. The y-axis represents the number of interview invitations received by the index student. The dashed line represents the situation when no students provide preferences. The dotted line represents only the index student providing preferences. The dash-dot line represents all students providing preferences except for the index student. The solid line represents all students providing preferences.

Figure 3

Distribution of Interviews by Preference Sharing^{a a} The contour lines indicate the number of interview invitations offered to each applicant based on their easy and hard scores relative to mean, ±2 standard deviations. With no preference provided, programs would offer interviews preferentially to a relatively small group of applicants with the highest easy scores. With preference, interviews were distributed more widely and more applicants with higher hard scores were included; this increase is seen within the area of the dotted polygon in the preferences plot. The applicants contained within the area of the dotted polygon on the (+) preference plot are 1 and 2 standard deviations above the mean in the hard category.