LASSO-based machine learning models for the prediction of central lymph node metastasis in clinically negative patients with papillary thyroid carcinoma

Abstract

Background: The presence of central lymph node metastasis (CLNM) is crucial for surgical decision-making in clinical N0 (cN0) papillary thyroid carcinoma (PTC) patients. We aimed to develop and validate machine learning (ML) algorithms-based models for predicting the risk of CLNM in cN0 patients.

Methods: A total of 1099 PTC patients with cN0 central neck from July 2019 to March 2022 at our institution were retrospectively analyzed. All patients were randomly split into the training dataset (70%) and the validation dataset (30%). Eight ML algorithms, including the Logistic Regression, Gradient Boosting Machine, Extreme Gradient Boosting (XGB), Random Forest (RF), Decision Tree, Neural Network, Support Vector Machine and Bayesian Network were used to evaluate the risk of CLNM. The performance of ML models was evaluated by the area under curve (AUC), sensitivity, specificity, and decision curve analysis (DCA).

Results: We firstly used the LASSO Logistic regression method to select the most relevant factors for predicting CLNM. The AUC of XGB was slightly higher than RF (0.907 and 0.902, respectively). According to DCA, RF model significantly outperformed XGB model at most threshold points and was therefore used to develop the predictive model. The diagnostic performance of RF algorithm was dependent on the following nine top-rank variables: size, margin, extrathyroidal extension, sex, echogenic foci, shape, number, lateral lymph node metastasis and chronic lymphocytic thyroiditis.

Conclusion: By incorporating clinicopathological and sonographic characteristics, we developed ML-based models, suggesting that this non-invasive method can be applied to facilitate individualized prediction of occult CLNM in cN0 central neck PTC patients.

Keywords: central lymph node metastasis; machine learning; papillary thyroid carcinoma; prediction model; random forest.

Figure 1

Selection of significant parameters in clinicopathologic variables in the training set. The values of the coefficients and the corresponding lambda values, each curve represents each feature in the model.

Figure 2

The mixed ROC curves of the eight machine learning models for prediction of CLNM. (A) The mixed ROC curves in the training cohort; (B) The mixed ROC curves in the validation cohort. ROC, receiver operating characteristic; CLNM, Central lymph node metastasis; LR, Logistic Regression; GBM, Gradient Boosting Machine; XGB, Extreme Gradient Boosting; RF, Random Forest; DT, Decision Tree; NNET, Neural Network; SVM, Support Vector Machine; BN, Bayesian Network.

Figure 3

The mixed Lift curves of the eight machine learning models for prediction of CLNM. The drawing process of the Lift curve is similar to the ROC curve, the difference is that the Lift value and the robust plane pose change in opposite directions, forming the opposite form of the Lift curve and the ROC curve. (A) The mixed Lift curves in the training cohort; (B) The mixed Lift curves in the validation cohort. CLNM, Central lymph node metastasis; ROC, receiver operating characteristic; LR, Logistic Regression; GBM, Gradient Boosting Machine; XGB, Extreme Gradient Boosting; RF, Random Forest; DT, Decision Tree; NNET, Neural Network; SVM, Support Vector Machine; BN, Bayesian Network.

Figure 4

Decision curve for predictive models based on machine learning models for prediction of CLNM. (A) The decision curve in the training cohort; (B) The decision curve in the validation cohort. CLNM, Central lymph node metastasis; LR, Logistic Regression; GBM, Gradient Boosting Machine; XGB, Extreme Gradient Boosting; RF, Random Forest; DT, Decision Tree; NNET, Neural Network; SVM, Support Vector Machine; BN, Bayesian Network.

Figure 5

Relative importance ranking of each input variable for prediction of CLNM in the machine learning models. (A) RF model; (B) XGB model; (C) GBM model; (D) NNET model. CLNM, Central lymph node metastasis; ETE, extrathyroidal extension; LLNM, lateral lymph node metastasis; CLT, chronic lymphocytic thyroiditis; RF, Random Forest; XGB, Extreme Gradient Boosting; GBM, Gradient Boosting Machine; NNET, Neural Network.

Figure 6

Predictive performance of the RF, XGB, GBM, NNET models with different numbers of variables. RF, Random Forest; XGB, Extreme Gradient Boosting; GBM, Gradient Boosting Machine; NNET, Neural Network.