The construction of HMME-PDT efficacy prediction model for port-wine stain based on machine learning algorithms

Abstract

Hematoporphyrin monomethyl ether-photodynamic therapy (HMME-PDT) is a safe and effective treatment for port-wine stain (PWS). Comprehensive methods for predicting HMME-PDT efficacy based on clinical factors are lacking. This study aims to develop and validate two machine learning models to predict the therapeutic effect of HMME-PDT for PWS. We conducted a retrospective study of 131 facial PWS patients treated with single HMME-PDT at the Second Xiangya Hospital from May 2022 to January 2025. The patients were divided into the training cohort and the validation cohort based on the order of their enrollment. Key clinical features were selected using recursive feature elimination (RFE). We developed and validated prediction models with Extreme Gradient Boosting (XGBoost) and Random Forest (RF) algorithms. Model performance was assessed using confusion matrix and evaluation metrics. RFE identified the top predictive factors: dermoscopy vascular pattern, immediate fluorescence intensity (IFI) after HMME-PDT, the facial port-wine stain area and severity index score, and age. In the training cohort, both models demonstrated strong predictive performance, with accuracies, F1 scores, and AUC values exceeding 0.8. The XGBoost model outperformed with an accuracy of 0.8750, F1 score of 0.8750, and AUC of 0.8636. In the validation cohort, XGBoost model achieved an accuracy and F1 score both greater than 0.73, with an AUC value of 0.7672. It had the better comprehensive performance. Our findings suggest these models are promising for predicting HMME-PDT efficacy in PWS. This is the first study to explore IFI after HMME-PDT in efficacy assessment.

Keywords: Efficacy prediction model; HMME-PDT; Immediate fluorescence intensity at the lesion site after HMME-PDT; Machine learning algorithms; Port-wine stain.

Fig. 1

Clinical pictures, dermoscopy and fluorescence imaging of port-wine stain (PWS)lesions before and after one session of hematoporphyrin monomethyl ether-photodynamic therapy (HMME-PDT). (a) A PWS patient that showed ≥ 25% improvement (effective). (a1) Before treatment and (a2) after HMME-PDT. (a3) Superficial type: dotted and globular, and short clubbed vessels were seen under dermoscopy before treatment. (a4) Immediate fluorescence intensity (IFI) at the lesion site after HMME-PDT where bright and homogeneous fluorescence was detected in the lesion. (b) A PWS patient that showed < 25% improvement (ineffective). (b1) Before treatment and (b2) after HMME-PDT. (b3) Deep type: reticular and linear vessels were seen under dermoscopy before treatment. (b4) Immediate fluorescence intensity at the lesion site after HMME-PDT where little fluorescence was detected in the lesion.

Fig. 2

Flowchart of the recursive feature elimination (RFE) with an Extreme Gradient Boosting (XGBoost) model as the estimator (XGBoost-RFE). The four curves in the figure represent the performance of the XGBoost-RFE model in 3-fold cross-validation and its mean performance. Based on the blue curve, the accuracy peaked when 4 or 5 features were retained.

Fig. 3

Receiver operating characteristic (ROC) curve of (a) XGBoost and (b) Random Forest (RF) on the testing dataset.

Fig. 4

Importance of each feature during the training process of the XGBoost model.

Fig. 5

Ranking of the average absolute SHapley Additive exPlanations (SHAP) values for each feature. The x-axis represents the absolute value of the average SHAP values for each feature, indicating the degree of its influence on the prediction results. Higher SHAP values indicate a greater impact of the feature on predicting a specific outcome.

Fig. 6

Overview of SHAP distribution for each feature in the XGBoost model for PWS. The x-axis represents the SHAP values attributed to each feature, and the y-axis lists the features in descending order of importance. Each point in the plot represents a patient, with the color indicating the feature value in the dataset: red denotes higher values, while blue represents lower values. Positive SHAP values suggest that the feature prompts the model to predict the therapy as effective, while negative SHAP values suggest the feature causes the model to predict the therapy as ineffective.

Fig. 7

SHAP distribution of key clinical features in the XGBoost efficacy prediction model for PWS. (a) Dermoscopy vascular pattern; (b) IFI at the lesion site after HMME-PDT; (c) The facial port-wine stain area and severity index (FSASI) score; (d) Age. The x-axis represents the statistical values of each feature (with the x-axis in (a) and (b) showing the encoded values of the features), and the y-axis shows the SHAP values attributed to each feature. Each point on the plot indicates the marginal contribution of a sample to the model’s prediction. The color gradient, ranging from blue to red, illustrates the value of another feature that may interact with the primary feature displayed on the x-axis. The color scale is consistent across all subplots, with blue indicating lower values and red indicating higher values of the secondary feature.