Machine Learning Algorithms for the Prediction of Central Lymph Node Metastasis in Patients With Papillary Thyroid Cancer

Abstract

Background: Central lymph node metastasis (CLNM) occurs frequently in patients with papillary thyroid cancer (PTC), but performing prophylactic central lymph node dissection is still controversial. There are no reliable models for predicting CLNM. This study aimed to develop predictive models for CLNM by machine learning (ML) algorithms.

Methods: Patients with PTC who underwent initial thyroid resection at our hospital between January 2018 and December 2019 were enrolled. A total of 22 variables, including clinical characteristics and ultrasonography (US) features, were used for conventional univariate and multivariate analysis and to construct ML-based models. A 5-fold cross validation strategy was used for validation and a feature selection approach was applied to identify risk factors.

Results: The areas under the receiver operating characteristic curve (AUC) of 7 models ranged from 0.680 to 0.731. All models performed significantly better than US (AUC=0.623) in predicting CLNM (P<0.05). In decision curve, most of the models also performed better than US. The gradient boosting decision tree model with 7 variables was identified as the best model because of its best performance in both ROC (AUC=0.731) and decision curves. Based on multivariate analysis and feature selection, young age, male sex, low serum thyroid peroxidase antibody and US features such as suspected lymph nodes, microcalcification and tumor size > 1.1 cm were the most contributing predictors for CLNM.

Conclusions: It is feasible to develop predictive models of CLNM in PTC patients by incorporating clinical characteristics and US features. The ML algorithm may be a useful tool for the prediction of lymph node metastasis in thyroid cancer.

Keywords: central lymph node metastasis; cross-validation; feature selection; machine learning; papillary thyroid cancer.

Figure 1

Receiver operating characteristic (ROC) curves of predictive models based on machine learning algorithms. US, ultrasonography; GBDT, gradient boosting decision tree; RFC, random forest classifier; AdaBoost, adaptive boosting; ANN, artificial neural network; MNB, multinomial naïve Bayes; XGBoost, extreme gradient boosting; DT, decision tree.

Figure 2

Decision curve for predictive models based on machine learning algorithms. US, ultrasonography; GBDT, gradient boosting decision tree; RFC, random forest classifier; AdaBoost, adaptive boosting; ANN, artificial neural network; MNB, multinomial naïve Bayes; XGBoost, extreme gradient boosting; DT, decision tree.

Figure 3

Ranks of the top 10 variables for the prediction of central lymph node metastasis. Variables were ranked using a classifier-specific importance evaluator based on machine learning algorithms. The variables are ordered according to the mean ranking of three potential models, which were GBDT, RFC and ANN. A lower rank represents more predictive importance. For example, age was ranked 1st, 3rd, and 10th in GBDT, ANN, and RFC, respectively. LNs, lymph nodes; Micro-Cal, microcalcification; TPO-Ab, thyroid peroxidase antibody; TSH, thyroid stimulating hormone; Ir. shape, irregular shape; Cap. invasion, capsular invasion; Hypo-echo, hypoechogenicity; GBDT, gradient boosting decision tree; RFC, random forest classifier; ANN, artificial neural network.

Figure 4

Predictive performance of the gradient boosting decision tree (GBDT) model with different numbers of variables. The AUC was the highest (0.731) with seven variables.