Machine Learning Analysis of Image Data Based on Detailed MR Image Reports for Nasopharyngeal Carcinoma Prognosis

Abstract

We aimed to assess the use of automatic machine learning (AutoML) algorithm based on magnetic resonance (MR) image data to assign prediction scores to patients with nasopharyngeal carcinoma (NPC). We also aimed to develop a 4-group classification system for NPC, superior to the current clinical staging system. Between January 2010 and January 2013, 792 patients with recent diagnosis of NPC, who had MR image data, were enrolled in the study. The AutoML algorithm was used and all statistical analyses were based on the 10-fold test. Primary endpoints included the probabilities of overall survival (OS), distant metastasis-free survival (DMFS), and local-region relapse-free survival (LRFS), and their sum was recorded as the final voting score, representative of progression-free survival (PFS) for each patient. The area under the receiver operating characteristic (ROC) curve generated from the MR image data-based model compared with the tumor, node, and metastasis (TNM) system-based model was 0.796 (P=0.008) for OS, 0.752 (P=0.053) for DMFS, and 0.721 (P=0.025) for LRFS. The Kaplan-Meier (KM) test values for II/I, III/II, IV/III groups in our new machine learning-based scoring system were 0.011, 0.010, and <0.001, respectively, whereas those for II/I, III/II, IV/III groups in the TNM/American Joint Committee on Cancer (AJCC) system were 0.118, 0.121, and <0.001, respectively. Significant differences were observed in the new machine learning-based scoring system analysis of each curve (P < 0.05), whereas the P values of curves obtained from the TNM/AJCC system, between II/I and III/II, were 0.118 and 0.121, respectively, without a significant difference. In conclusion, the AutoML algorithm demonstrated better prognostic performance than the TNM/AJCC system for NPC. The algorithm showed a good potential for clinical application and may aid in improving counseling and facilitate the personalized management of patients with NPC. The clinical application of our new scoring and staging system may significantly improve precision medicine.

Figure 1

Flow chart of feature selection used in this study. First step: feature selection to reduce features. Second step: AutoML is run. AutoML performs hyperparameter search (parameters such as tree and depth in the flow chart are representative examples) over a variety of H2O algorithms to deliver the best model. The hyperparameters of AutoML supported by grid search are listed in Supplementary . Abbreviations: mRMR, minimum redundancy maximum correlation; GLM, generalized linear model; XRT, extreme random tree; GBM, gradient boosting machine; RF, random forest; DL, deep learning.

Figure 2

Feature selection as performed by AutoML. (a) The important imaging findings of OS with 13 variables, ranked according to their importance, are listed; the best AUC is selected. (b) The important imaging findings of DMFS with 12 variables, ranked, are listed; the best AUC is selected. (c) The important imaging findings of LRFS with 11 variables, ranked, are listed; the best AUC is selected.

Figure 3

Receiver operating characteristics of Cox regression. (a) We used PFS fraction and PFS to establish the ROC curve, and the best cutoff value was 0.986 (highest Youden-index), which is regarded as the first cutoff value. We considered the first cutoff value over 0.986 for the new staging IV. Next, we used the remaining patients to draw the ROC curve and calculate the remaining two cutoff values. (b) The second cutoff value is not less than 0.643, which is in accordance with staging III. (c) The third cutoff value is not less than 0.270, which is in accordance with staging I and II. Stage IV PFS%≥0.986, Stage III PFS%≥0.643, Stage II PFS%≥0.270, and Stage I PFS% <0.270.

Figure 4

Analysis of Kaplan-Meier survival curve for the estimation of progression-free survival (PFS): (a) new scoring system (b) AJCC staging.