Deep pathomics: A new image-based tool for predicting response to treatment in stage III non-small cell lung cancer

Abstract

Despite the advantages offered by personalized treatments, there is presently no way to predict response to chemoradiotherapy in patients with non-small cell lung cancer (NSCLC). In this exploratory study, we investigated the application of deep learning techniques to histological tissue slides (deep pathomics), with the aim of predicting the response to therapy in stage III NSCLC. We evaluated 35 digitalized tissue slides (biopsies or surgical specimens) obtained from patients with stage IIIA or IIIB NSCLC. Patients were classified as responders (12/35, 34.7%) or non-responders (23/35, 65.7%) based on the target volume reduction shown on weekly CT scans performed during chemoradiation treatment. Digital tissue slides were tested by five pre-trained convolutional neural networks (CNNs)-AlexNet, VGG, MobileNet, GoogLeNet, and ResNet-using a leave-two patient-out cross validation approach, and we evaluated the networks' performances. GoogLeNet was globally found to be the best CNN, correctly classifying 8/12 responders and 10/11 non-responders. Moreover, Deep-Pathomics was found to be highly specific (TNr: 90.1) and quite sensitive (TPr: 0.75). Our data showed that AI could surpass the capabilities of all presently available diagnostic systems, supplying additional information beyond that currently obtainable in clinical practice. The ability to predict a patient's response to treatment could guide the development of new and more effective therapeutic AI-based approaches and could therefore be considered an effective and innovative step forward in personalised medicine.

Fig 1. During the first phase, the data modeling randomly selected two tissue slides as the test set: One from the responder group (red tissue slides), and the other from the non-responder group (blue tissue slides).

In the figure, slides 3 and 6 were selected as the test set from the responder and non-responder groups, respectively. The 33 remaining patients were used as the training set, to feed the machine learning system. All features from the 33 patients were extracted and analyzed to train the artificial intelligence. During the second phase, the convolutional neural networks (CNNs) ignored the belonging group of the two patients from the test set (gray tissue slides), and classified them as responders or non-responders based on their similarity to the training set tissue slides.

Fig 2

A) Tissue slide of a spindle cell carcinoma of the lung. Three regions of interest (ROIs) were selected (blue squares) based on tumor heterogeneity. B–D) Close-up views of tumor cells within desmoplastic fibrosis (B), areas of necrosis within tumor aggregates (C), and prominent inflammation surrounding tumor cells (D).

Fig 3. The sequence of crops and patches extracted from a single tumor.

Each group of cancer cells is contoured, defining 9 crops (areas contoured in red) within a single region of interest (ROI; blue square).

Fig 4. AlexNet structure with key workflow.

Fig 5. VGG structure with key workflow.

Fig 6. MobileNet structure with key workflow.

Fig 7. GoogLeNet structure with key workflow.

Fig 8. GoogLeNet input layer.

Fig 9. ResNet structure with workflow.

Fig 10. ResNet input layer.

Fig 11. ResNet residual block.

Fig 12. ResNet residual learning layer.

Fig 13. Depiction of the utilised pipeline.

it consists of the following parts:

1. Image acquisition: several images are acquired for each patient;

2. Images segmentation: each image undergoes a manual segmentation process by an experienced physician to extract only the pixels containing the cancer cells;

3. Model training: the selected pre-trained model is fine-tuned on all the training crops belonging to the selected fold. The validation technique is the leave-one-patient-out and ensures that all the crops extracted from a patient can compose either the training or the test set, thus preventing the model from being biased;

4. Model test: all the crops in the test set of the selected fold are analysed by the trained model, which returns a prediction. In order to maintain balance between classes each test set always consists of the crops from two patients coming with different labels;

5. Performance evaluation: once the model has predicted the outcome of all crops from each test fold, a global performance evaluation step calculates the metrics needed to compare the models

.

Fig 14. Two tissue slides that were failed by all the convolutional neural networks (CNNs).

A) Whole-slide image (WSI) of a responder patient affected by adenocarcinoma, who was incorrectly classified as a non-responder by all the CNNs (false negative). Scale bar: 5 mm. B) Close-up view showing fragmented and stretched tumor glands. Scale bar: 250 μm. C) WSI of a non-responder patient affected by spindle cell carcinoma, who was incorrectly classified as a responder by all the CNNs (false positive). Scale bar: 2.5 mm. D) Close-up view showing tumor infiltrating the submucosal stroma. Scale bar: 250 μm.

Fig 15. The evolution of training loss across epochs is delineated for the various CNN architectures.

The solid line denotes the mean value encompassing the range of experiments, while the shaded colored region signifies the span of standard deviation.

Fig 16. The flowchart outlines the potential practical implementation of deep-pathomics.

Following tissue collection, either through biopsy or surgery, and subsequent histological diagnosis, the tumor tissue slides undergo digitalization via Whole Slide Imaging, without further additional tissue consuming; therefore, the residual tumor tissue will be available for molecular testing and immunohistochemistry. Tumor tissue slides from eligible patients for chemoradiation will undergo assessment using Deep-Pathomics. Based on this analysis, different personalized therapeutic strategies can be employed accordingly. The illustration depicts two distinct patients affected by Stage IIIB adenocarcinoma of the lung. Despite the same diagnosis, these patients underwent different treatment approaches as determined by Deep-Pathomics analysis.