ConvPath: A software tool for lung adenocarcinoma digital pathological image analysis aided by a convolutional neural network

Abstract

Background: The spatial distributions of different types of cells could reveal a cancer cell's growth pattern, its relationships with the tumor microenvironment and the immune response of the body, all of which represent key "hallmarks of cancer". However, the process by which pathologists manually recognize and localize all the cells in pathology slides is extremely labor intensive and error prone.

Methods: In this study, we developed an automated cell type classification pipeline, ConvPath, which includes nuclei segmentation, convolutional neural network-based tumor cell, stromal cell, and lymphocyte classification, and extraction of tumor microenvironment-related features for lung cancer pathology images. To facilitate users in leveraging this pipeline for their research, all source scripts for ConvPath software are available at https://qbrc.swmed.edu/projects/cnn/.

Findings: The overall classification accuracy was 92.9% and 90.1% in training and independent testing datasets, respectively. By identifying cells and classifying cell types, this pipeline can convert a pathology image into a "spatial map" of tumor, stromal and lymphocyte cells. From this spatial map, we can extract features that characterize the tumor micro-environment. Based on these features, we developed an image feature-based prognostic model and validated the model in two independent cohorts. The predicted risk group serves as an independent prognostic factor, after adjusting for clinical variables that include age, gender, smoking status, and stage.

Interpretation: The analysis pipeline developed in this study could convert the pathology image into a "spatial map" of tumor cells, stromal cells and lymphocytes. This could greatly facilitate and empower comprehensive analysis of the spatial organization of cells, as well as their roles in tumor progression and metastasis.

Keywords: Cell distribution and interaction; Convolutional neural network; Deep learning; Lung adenocarcinoma; Pathology image; Prognosis.

Fig. 1

Flow chart of ConvPath-aided pathological image analysis. CHCAMS, National Cancer Center/Cancer Hospital of Chinese Academy of Medical Sciences, China; CI, confidence interval; HR, hazard ratio; TCGA, The Cancer Genome Atlas.

Fig. 2

Image preprocessing step of the ConvPath software. (a) Selection of regions of interest (ROIs) in whole pathological imaging slides. (b) Image segmentation pipeline to extract cell-centered image patches from selected ROIs.

Fig. 3

Cell type recognition step of the ConvPath software. (a) Schema and structure of the convolutional neural network (CNN) to recognize the types of cells in the centers of image patches. (b) Confusion matrix of internal testing results of CNN on the NLST and TCGA training image slides. Prediction accuracies are calculated based on 3996 image patches for each cell type. (c) Confusion matrix of independent testing results of CNN on image patches of the SPORE dataset. Prediction accuracies are calculated based on 8245 lymphocyte, 2211 stroma, and 6836 tumor patches.

Fig. 4

Feature extraction step of the ConvPath software. (a) A zoomed-in part of a sampling region (Supplemental Figure 3) in which cell nuclei centroids are labeled with predicted cell types. Green, stroma; cyan, lymphocyte; yellow, tumor. (b) Cell type region detection using a kernel smoothing algorithm for the sampling region shown in Supplemental Figure 3. Area and perimeters are evaluated for regions of tumor, stroma, and lymphocyte.

Fig. 5

Application of the prognostic model to independent datasets. (a, b) Validation of the prognostic model in the TCGA overall survival data (a, log rank test, p = 0.0047) and the CHCAMS recurrence data (b, log rank test, p = 0.030). (c) Boxplot for the distribution of predicted risk scores in the 5 histological subtypes of lung adenocarcinoma for the CHCAMS dataset patients. Jonckheere-Terpstra k-sample test, p = 0.0039. The boxes and whiskers show the lower (Q1) and upper (Q3) quartiles and the median for each histological subtype.