Prediction of Medical Disputes Between Health Care Workers and Patients in Terms of Hospital Legal Construction Using Machine Learning Techniques: Externally Validated Cross-Sectional Study

Abstract

Background: Medical disputes are a global public health issue that is receiving increasing attention. However, studies investigating the relationship between hospital legal construction and medical disputes are scarce. The development of a multicenter model incorporating machine learning (ML) techniques for the individualized prediction of medical disputes would be beneficial for medical workers.

Objective: This study aimed to identify predictors related to medical disputes from the perspective of hospital legal construction and the use of ML techniques to build models for predicting the risk of medical disputes.

Methods: This study enrolled 38,053 medical workers from 130 tertiary hospitals in Hunan province, China. The participants were randomly divided into a training cohort (34,286/38,053, 90.1%) and an internal validation cohort (3767/38,053, 9.9%). Medical workers from 87 tertiary hospitals in Beijing were included in an external validation cohort (26,285/26,285, 100%). This study used logistic regression and 5 ML techniques: decision tree, random forest, support vector machine, gradient boosting decision tree (GBDT), and deep neural network. In total, 12 metrics, including discrimination and calibration, were used for performance evaluation. A scoring system was developed to select the optimal model. Shapley additive explanations was used to generate the importance coefficients for characteristics. To promote the clinical practice of our proposed optimal model, reclassification of patients was performed, and a web-based app for medical dispute prediction was created, which can be easily accessed by the public.

Results: Medical disputes occurred among 46.06% (17,527/38,053) of the medical workers in Hunan province, China. Among the 26 clinical characteristics, multivariate analysis demonstrated that 18 characteristics were significantly associated with medical disputes, and these characteristics were used for ML model development. Among the ML techniques, GBDT was identified as the optimal model, demonstrating the lowest Brier score (0.205), highest area under the receiver operating characteristic curve (0.738, 95% CI 0.722-0.754), and the largest discrimination slope (0.172) and Youden index (1.355). In addition, it achieved the highest metrics score (63 points), followed by deep neural network (46 points) and random forest (45 points), in the internal validation set. In the external validation set, GBDT still performed comparably, achieving the second highest metrics score (52 points). The high-risk group had more than twice the odds of experiencing medical disputes compared with the low-risk group.

Conclusions: We established a prediction model to stratify medical workers into different risk groups for encountering medical disputes. Among the 5 ML models, GBDT demonstrated the optimal comprehensive performance and was used to construct the web-based app. Our proposed model can serve as a useful tool for identifying medical workers at high risk of medical disputes. We believe that preventive strategies should be implemented for the high-risk group.

Keywords: hospital legal construction; machine learning; medical disputes; medical workers; multicenter analysis.

Figure 1

Flowchart outlining the participants’ enrollment and study profile. Participants from the Hunan province were randomly divided into a training set and an internal validation set. The logistic regression model and 5 machine learning models were trained and optimized in the training set, and the internal validation set was used to internally validate the models. Participants from Beijing served as the external validation set. A scoring system was developed after incorporating 21 metrics to assess the prediction performance of the models.

Figure 2

Heat map for the prediction performance of each model in the (A) internal validation set and (B) external validation set after evaluation using a scoring system. Cyan-colored boxes indicate a low value, whereas orange-colored boxes represent a relatively high value. The total score was calculated as the sum of the values from the 12 metrics, and a higher total score indicates better prediction performance. AUC: area under the curve; NPV: negative predictive value.

Figure 3

Probability curve for (A) logistic regression, (B) decision tree, (C) random forest, (D) support vector machine, (E) gradient boosting decision tree, and (F) deep neural network in the internal validation set. The probability curve was drawn with predicted probability against density. Cyan color indicates participants without medical disputes, and orange color indicates participants with medical disputes.

Figure 4

Model explainability using local interpretable model-agnostic explanation: (A) a true positive case and (B) a true negative case. In the first case, the study depicted a specific individual with a high probability of experiencing medical dispute (64%), whereas the second case showed low risk of medical dispute (21%). Features with an orange bar imply contributory elements to increase medical dispute, whereas those with a cyan bar indicate protective features.