AI-guided histopathology predicts brain metastasis in lung cancer patients

Abstract

Brain metastases can occur in nearly half of patients with early and locally advanced (stage I-III) non-small cell lung cancer (NSCLC). There are no reliable histopathologic or molecular means to identify those who are likely to develop brain metastases. We sought to determine if deep learning (DL) could be applied to routine H&E-stained primary tumor tissue sections from stage I-III NSCLC patients to predict the development of brain metastasis. Diagnostic slides from 158 patients with stage I-III NSCLC followed for at least 5 years for the development of brain metastases (Met⁺, 65 patients) versus no progression (Met^-, 93 patients) were subjected to whole-slide imaging. Three separate iterations were performed by first selecting 118 cases (45 Met⁺, 73 Met^-) to train and validate the DL algorithm, while 40 separate cases (20 Met⁺, 20 Met^-) were used as the test set. The DL algorithm results were compared to a blinded review by four expert pathologists. The DL-based algorithm was able to distinguish the eventual development of brain metastases with an accuracy of 87% (p < 0.0001) compared with an average of 57.3% by the four pathologists and appears to be particularly useful in predicting brain metastases in stage I patients. The DL algorithm appears to focus on a complex set of histologic features. DL-based algorithms using routine H&E-stained slides may identify patients who are likely to develop brain metastases from those who will remain disease free over extended (>5 year) follow-up and may thus be spared systemic therapy. © 2024 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Keywords: artificial intelligence; brain metastasis; deep learning; digital pathology; non‐small cell lung cancer.

Figure 1.. Data processing pipeline.

(A) A single, representative hematoxylin and eosin (H/E) stained slide of a surgically resected primary NSCLC tumor block was obtained from 158 patient cases and scanned at 40X magnification. Each scan file was coded and linked to outcome and pathology data, but blinded to both the DL team and review pathologists until predictions were finalized. From each whole slide scan, regions of high tumor cellularity and surrounding tumor microenvironment were annotated by one reviewing pathologist. Regions outside of the tumor bed as well as areas of blank glass were masked. (B) 1,000 non-overlapping image tiles from the region of interest (ROI) of each scan file were selected at random. Tiles were subjected to color normalization and randomized in cropping and orientation to create a data augmentation step. (C) All tiles in the training set were shuffled and fed to the convolution neural network with the ResNet-18 backbone pre-trained on ImageNet, with a linear layer and sigmoid activation for model optimization. In the testing process, the weights in the model were all frozen. A median-pooling function was used to compute the final risk assessment from the collective image tiles of each patient.

Figure 2.. Deep learning study design.

The cases (slide images) were arbitrarily coded from 1 to 158 (shown in top grayscale bar). The cases were randomized (shown in randomized grayscale bars) and split in three different partitions to create experiments 1 to 3. Each experiment utilized a different training set of 30 Met⁺ (orange) and 58 Met^– (purple) tumor images and a validation set of 15 Met⁺ and 15 Met^– images. The training-validation was performed using a three-fold cross-validation. Each subsequent set-aside testing set was composed of 20 Met⁺ and 20 Met^– case images. The testing sets for experiments 1 to 3 in total represented ~75% of the entire 158 case cohort.

Figure 3.. DL versus pathologist prediction of progression.

(A) ROC curves generated from three experimental rounds of training and validation using 3-fold cross validation. For comparison, the ROC curve generated from the identical process using random phenotype assignment is shown by the green dotted line. The threshold for calculating testing set prediction accuracies in each experimental session is indicated with a star. For comparison, the sensitivity and specificity for prediction by four independent pathologists are shown as points. (B) The bar plot shows testing accuracies for deep learning model (DL), four independent pathologist reviews (PA, PB, PC, PD, and average- AP), and the random classifier (RC) across three experiments. For each pathologist, the predictions were performed using the same slides in three testing sets implemented in the DL model, resulting in three accuracy scores. The error bars represent one standard deviation from the mean value. ‘ns’ stands for not significant; ‘***’ and ‘****’ indicate p<0.0001 and p<0.00001, respectively. (C) Confusion matrix of DL model performance combined over three experiments. (D) Confusion matrix of averaged pathologists’ evaluation on the three testing sets.

Figure 4.. Model prediction maps.

(A, C) Prediction maps of non-progression cases (Met^–); (B, D) prediction maps of progression cases (Met⁺). Dark orange pixels in the prediction maps indicate image features scored with a high progression risk, and the deep purple pixels represent features scored as a low progression risk. (A1-A3), (B1-B3), (C1-C3), (D1-D3) are corresponding high-power images of the H&E-stained regions indicated within each prediction map. (A1), (C1), and (D2) show high power H&E stained images of regions with an incorrectly predicted risk.