Development and validation of an early diagnosis model for bone metastasis in non-small cell lung cancer based on serological characteristics of the bone metastasis mechanism

Abstract

Background: Bone metastasis significantly impact the prognosis of non-small cell lung cancer (NSCLC) patients, reducing their quality of life and shortening their survival. Currently, there are no effective tools for the diagnosis and risk assessment of early bone metastasis in NSCLC patients. This study employed machine learning to analyze serum indicators that are closely associated with bone metastasis, aiming to construct a model for the timely detection and prognostic evaluation of bone metastasis in NSCLC patients.

Methods: The derivation cohort consisted of 664 individuals with stage IV NSCLC, diagnosed between 2015 and 2018. The variables considered in this study included age, sex, and 18 specific serum indicators that have been linked to the occurrence of bone metastasis in NSCLC. Variable selection used multivariate logistic regression analysis and Lasso regression analysis. Six machine learning methods were utilized to develop a bone metastasis diagnostic model, assessed with Area Under the Curve (AUC), Decision Curve Analysis (DCA), sensitivity, specificity, and validation cohorts. External validation used 113 NSCLC patients from the Medical Alliance (2019-2020). Furthermore, a prospective validation study was conducted on a cohort of 316 patients (2019-2020) who were devoid of bone metastasis, and followed-up for at least two years to assess the predictive capabilities of this model. The model's prognostic value was evaluated using Kaplan-Meier survival curves.

Findings: Through variable selection, 11 serum indictors were identified as independent predictive factors for NSCLC bone metastasis. Six machine learning models were developed using age, sex, and these serum indicators. A random forest (RF) model demonstrated strong performance during the training and internal validation cohorts, achieving an AUC of 0.98 (95% CI 0.95-0.99) for internal validation. External validation further confirmed the RF model's effectiveness, yielding an AUC of 0.97 (95% CI 0.94-0.99). The calibration curves demonstrated a high level of concordance between the anticipated risk and the observed risk of the RF model. Prospective validation revealed that the RF model could predict the occurrence of bone metastasis approximately 10.27 ± 3.58 months in advance, according to the results of the SPECT. An online computing platform (https://bonemetastasis.shinyapps.io/shiny_cls_1model/) for this RF model is publicly available and free-to-use by doctors and patients.

Interpretation: This study innovatively employs age, gender, and 11 serological markers closely related to the mechanism of bone metastasis to construct an RF model, providing a reliable tool for the early screening and prognostic assessment of bone metastasis in NSCLC patients. However, as an exploratory study, the findings require further validation through large-scale, multicenter prospective studies.

Funding: This work is supported by the National Natural Science Foundation of China (NO.81974315); Shanghai Municipal Science and Technology Commission Medical Innovation Research Project (NO.20Y11903300); Shanghai Municipal Health Commission Health Industry Clinical Research Youth Program (NO.20204Y034).

Keywords: Bone metastasis; Early diagnosis; Machine learning; Model interpretability; Risk assessment.

Fig. 1

Flow diagram of the study population. Abbreviations: BM: bone metastasis; NBM: none-bone metastasis; NSCLC: Non-Small Cell Lung Cancer.

Fig. 2

Multivariate logistic regression analysis and variable selection process of Lasso regression. A) Multiple logistic regression analysis of the risk scores and clinical parameters. B and C) Feature visualization of Lasso regression feature factor analysis and Shapley value. Abbreviations: shay.abs.mean: Shapley additive explanations absolute mean value; Ca: Calcium; P: Phosphorus; PTH: Parathyroid Hormone; CT: Calcitonin; BAP: Bone specific alkaline phosphatase; tP1NP: Total type I procollagen amino-terminal peptide; OPG: Ostoeprotegerin; NMID: N-Mid Osteocalcin; β-CTx: β-type I procollagen carboxy-terminal peptide; FT3: Free thiiodothyronine; FT4: Free thyroxine; TSH: Thyroid Stimulating Hormone; PTHRP: Parathyroid hormone-related protein; IL-6: Interleukin-6; K: Potassium; Na: Sodium; Mg: Magnesium; Cl: Chloridion.

Fig. 3

Performance comparison of six models on the internal validation cohort. A) ROC curves for each model in the validation cohort; B) 95% confidence intervals for ROC-AUC obtained through cross-validation for each model; C) Parallel line graph of the evaluation metrics for each model; D) DCA curves for each model. Abbreviations: DT: Decision tree model; Logistic: Logistic regression model; MLP: Multilayer perceptron model; RF: Random forest model; SVM: Support vector machine model; Xgboost: Extreme gradient boosting model; f-means: F1 score; J-index: Youden index; kap: Kappa coefficient; mcc: Matthews correlation coefficient; NPV: Negative Predictive Value; PPV: Positive Predictive Value; CV ROC-AUC: cross-validation Area Under the Receiver Operating Characteristic Curve; Sens: Sensitivity; Spec: Specificity.

Fig. 4

Performance comparison of six models on the external validation cohort. A) ROC-AUC with 5-fold cross-validation; B) ROC curve on the validation cohort; C) Calibration plots (Reliability curve, dashed line represents perfectly calibrated) comparing the predictive performance of six models using external validation data from 113 cases. Abbreviations: DT: Decision tree model; Logistic: Logistic regression model; MLP: Multilayer perceptron model; RF: Random forest model; SVM: Support vector machine model; Xgboost: Extreme gradient boosting model; ROC-AUC: Area Under the Receiver Operating Characteristic Curve; ROC: Receiver Operating Characteristic Curve.

Fig. 5

RF model based on the SHAP algorithm. A) Feature attributes in the black-box model. Each line represents a feature, and the x-axis represents SHAP values, indicating the impact of the feature on the outcome. Each point represents a sample. The redder the colour, the larger the feature value; the bluer the colour, the smaller the feature value; B) Feature importance ranking indicated by SHAP. Abbreviations: AUC: Area Under the Curve; RF: Random forest; SHAP: Shapley additive explanations; Ca: Calcium; P: Phosphorus; CT: Calcitonin; BAP: Bone specific alkaline phosphatase; tP1NP: Total type I procollagen amino-terminal peptide; β-CTx: β-type I procollagen carboxy-terminal peptide; FT4: Free thyroxine; PTHRP: Parathyroid hormone-related protein; K: Potassium; Na: Sodium; Mg: Magnesium.

Fig. 6

Kaplan–Meier survival analysis of six models on the prospective validation cohort. A) DT: Decision tree model; B) Logistic: Logistic regression model; C) MLP: Multilayer perceptron model; D) RF: Random forest model; E) SVM: Support vector machine model; F) Xgboost: Extreme gradient boosting model. Abbreviations: HR: Hazard Ratio; Log-rank: Log–Rank test.

Fig. 7

Online computing platform presentation of the optimal RF model. Abbreviations: RF: Random forest; Ca: Calcium; P: Phosphorus; CT: Calcitonin; BAP: Bone specific alkaline phosphatase; tP1NP: Total type I procollagen amino-terminal peptide; CTx: β-type I procollagen carboxy-terminal peptide; FT4: Free thyroxine; PTHRP: Parathyroid hormone-related protein; K: Potassium; Na: Sodium; Mg: Magnesium; SHAP value: Shapley additive explanations absolute mean value; Pred: Prediction; Prob: Probability; BM: bone metastasis.