Predicting central lymph node metastasis in papillary thyroid microcarcinoma: a breakthrough with interpretable machine learning

Abstract

Objective: To develop and validate an interpretable machine learning (ML) model for the preoperative prediction of central lymph node metastasis (CLNM) in papillary thyroid microcarcinoma (PTMC).

Methods: From December 2016 to December 2023, we retrospectively analyzed 710 PTMC patients who underwent thyroidectomies. Feature selection was conducted using the least absolute shrinkage and selection operator (LASSO) regression method, alongside the Support Vector Machine-Recursive Feature Elimination (SVM-RFE) algorithm in conjunction with multivariate logistic regression. Eight ML algorithms, namely Decision Tree, Random Forest (RF), K-nearest neighbors, Support vector machine, Extreme Gradient Boosting, Naive Bayes, Logistic regression, and Light Gradient Boosting machine, were developed for the prediction of CLNM. The performance of these models was evaluated using area under the receiver operating characteristic curve (AUC), decision curve analysis (DCA), sensitivity, specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV), and F1 scores. Additionally, the Shapley Additive Explanation (SHAP) algorithm was utilized to clarify the results of the optimal ML model.

Results: The results indicated that 32.95% of the patients (234/710) presented with CLNM. Tumor diameter, multifocality, lymph nodes identified via ultrasound (US-LN), and extrathyroidal extension (ETE) were identified as independent predictors of CLNM. The RF model achieved the highest performance in the validation set with an AUC of 0.893(95%CI: 0.846-0.940), accuracy of 0.832, sensitivity of 0.764, specificity of 0.866, PPV of 0.743, NPV of 0.879, and F1-score of 0.753. Furthermore, the DCA demonstrated that the RF model exhibited a superior clinical net benefit.

Conclusion: Our model predicted the risk of CLNM in PTMC patients with high accuracy preoperatively.

Keywords: SHapley Additive exPlanation; central lymph node metastasis; diagnostic imaging; machine learning; papillary thyroid microcarcinoma.

Figure 1

Patient flowchart for this study. CLND, central lymph node dissection; PTC, papillary thyroid carcinoma; PTMC, papillary thyroid microcarcinoma; CLNM, central lymph node metastasis.

Figure 2

Artificial intelligence workflow and study flowchart. DT, decision tree; RF, random forest; SVM, support vector machine; XGBoost, Extreme Gradient Boosting; KNN, k-nearest neighbors; LR, logistic regression; LightGBM, light gradient boosting machine; NBM, naive bayes model; SHAP, Shapley Additive Explanation.

Figure 3

Feature selection was performed by the Least Absolute Shrinkage and Selection Operator (LASSO) regression and Support Vector Machine Recursive Feature Elimination (SVM-RFE). (A) Coefficients derived from LASSO regression. (B) The range of optimal values was identified by the LASSO model. (C) Four optimal features were chosen by the LASSO. (D) Five optimal features were chosen by the SVW-RFE. US-LN, cervical lymph nodes status based on ultrasound; ETE, extrathyroidal extension; TSH, thyroid-stimulating hormone.

Figure 4

Presents a comparative analysis of various machine learning models employed for the prediction of cervical lymph node metastasis (CLNM) in papillary thyroid microcarcinoma (PTMC) patients. (A) ROC curves of the eight model in the validation set. (B) ROC curves evaluate the RF model and US-LN through AUC scores. (C) Decision curve analysis in the validation set. ROC, receiver operating characteristic; AUC, area under the ROC curve; DT, decision tree; KNN, k-nearest neighbors; LightGBM, light gradient boosting machine; LR, logistic regression; NBM, naive bayes model; RF, random forest; SVW, support vector machine; XGBoost, Extreme Gradient Boosting.

Figure 5

Effectiveness evaluation of RF prediction models (A) Confusion matrix of RF in the training set. (B) Confusion matrix of the RF model in the validation set. X-axis represents the model prediction, y-axis represents the real situation, and the values in the box are the number of samples. (C) Standard deviation of feature importance in the RF model.

Figure 6

Shapley Additive Explanation (SHAP) of the model. (A) Summary plots for the validation sets with associated SHAP values. Each point represents a SHAP value for a patient’s characteristic. (B) SHAP force plot for a PTMC patient without CLMN. (C) SHAP force plot for a PTMC patient with CLMN. US-LN, cervical lymph nodes status based on ultrasound; ETE, extrathyroidal extension.