Predicting physician departure with machine learning on EHR use patterns: A longitudinal cohort from a large multi-specialty ambulatory practice

Abstract

Physician turnover places a heavy burden on the healthcare industry, patients, physicians, and their families. Having a mechanism in place to identify physicians at risk for departure could help target appropriate interventions that prevent departure. We have collected physician characteristics, electronic health record (EHR) use patterns, and clinical productivity data from a large ambulatory based practice of non-teaching physicians to build a predictive model. We use several techniques to identify possible intervenable variables. Specifically, we used gradient boosted trees to predict the probability of a physician departing within an interval of 6 months. Several variables significantly contributed to predicting physician departure including tenure (time since hiring date), panel complexity, physician demand, physician age, inbox, and documentation time. These variables were identified by training, validating, and testing the model followed by computing SHAP (SHapley Additive exPlanation) values to investigate which variables influence the model's prediction the most. We found these top variables to have large interactions with other variables indicating their importance. Since these variables may be predictive of physician departure, they could prove useful to identify at risk physicians such who would benefit from targeted interventions.

Fig 1. Primary model performance on unseen test data.

Receiver Operating Characteristic curve and Precision Recall Curve with confidence bands and computed Area Under the Curve values. Diagonal dashed line shows the curve expected from a no-skill classifier.

Fig 2. Shapley Additive Explanations (SHAP) analysis Beeswarm Plot showing the 10 top features contributing to physician departure.

Each dot represents a physician-month. Positive SHAP values (right of 0.0 vertical line) indicate the feature increased the individual physician’s monthly risk of departure. Actual feature values are color-coded with high feature values indicated in red, low values in blue and null values in gray.

Fig 3. Dependence plots for the features with greatest average absolute SHAP values.

Feature value is displayed on the x-axis and the associated SHAP value is displayed on the y-axis. SHAP values greater than zero denote an increase in risk. Color shows the value of the secondary feature estimated to have the strongest interaction with the primary feature.

Fig 4. SHAP values for interactions between tenure and EHR use metrics.

For each plot, the SHAP value for the interaction between tenure and an EHR use metric is shown as a function of tenure. The x-axis displays the value of tenure, the y-axis shows the SHAP value, and the color encodes the value of the EHR use metric. SHAP values for interactions above zero denote an increase in risk.

Fig 5. Features with greatest SHAP value change when physician classification changed.

The top 15 values for average per-physician change in log-odds contribution for each feature and pairwise interaction as estimated by SHAP are displayed.