Automated deep learning-based assessment of tumour-infiltrating lymphocyte density determines prognosis in colorectal cancer

Abstract

Background: The presence of tumour-infiltrating lymphocytes (TILs) is a well-established prognostic biomarker across multiple cancer types, with higher TIL counts being associated with lower recurrence rates and improved patient survival. We aimed to examine whether an automated intraepithelial TIL (iTIL) assessment could stratify patients by risk, with the ability to generalise across independent patient cohorts, using routine H&E slides of colorectal cancer (CRC). To our knowledge, no other existing fully automated iTIL system has demonstrated this capability.

Methods: An automated method employing deep neural networks was developed to enumerate iTILs in H&E slides of CRC. The method was applied to a Stage III discovery cohort (n = 353) to identify an optimal threshold of 17 iTILs per-mm² tumour for stratifying relapse-free survival. Using this threshold, patients from two independent Stage II-III validation cohorts (n = 1070, n = 885) were classified as "TIL-High" or "TIL-Low".

Results: Significant stratification was observed in terms of overall survival for a combined validation cohort univariate (HR 1.67, 95%CI 1.39-2.00; p < 0.001) and multivariate (HR 1.37, 95%CI 1.13-1.66; p = 0.001) analysis. Our iTIL classifier was an independent prognostic factor within proficient DNA mismatch repair (pMMR) Stage II CRC cases with clinical high-risk features. Of these, those classified as TIL-High had outcomes similar to pMMR clinical low risk cases, and those classified TIL-Low had significantly poorer outcomes (univariate HR 2.38, 95%CI 1.57-3.61; p < 0.001, multivariate HR 2.17, 95%CI 1.42-3.33; p < 0.001).

Conclusions: Our deep learning method is the first fully automated system to stratify patient outcome by analysing TILs in H&E slides of CRC, that has shown generalisation capabilities across multiple independent cohorts.

Keywords: Cell detection; Computational pathology; Image analysis; Tissue segmentation.

Fig. 1

(A) Given a whole slide image, our proposed AI iTIL scoring method first performs coarse semantic segmentation to identify broadly cancerous tissue. Within the cancerous region, one fine-grained model segments areas of tumour, stroma, and necrosis and another detects TILs. A summary iTIL score is calculated from the median quantification of TIL density associated with tumour regions, then thresholded to stratify patient outcome. (B) 512 × 512px regions fed to each model with different amounts of context, representing tissue areas of 2.048mm² at 4MPP, 256 µm² at 0.5 MPP, and 128 µm² at 0.25 MPP respectively. (C) Given an input image, our TIL model generates a heatmap consisting of circular, blob-like areas which correspond to the likelihood that each pixel belongs to a TIL. Using blob detection, we extract a set of point coordinates (shown with yellow dots) corresponding to TIL locations, which rejects the detection of TILs in areas of low likelihood (seen as faint blobs in the heatmap)

Fig. 2

Predictions on test set regions from the broad tumour (A) (average F1: 0.9819), TSN (B) (average F1: 0.6460), and TIL (C) (F1: 0.9231) models. Broad tumour: green represents background and normal tissue, red represents cancerous tissue. TSN: Green represents stroma, red represents tumour

Fig. 3

Kaplan-Meier curves illustrating stratification by the AI-generated iTIL score. (A) Combined cohort 5-Year OS, (B) RNSH-CRC 5-Year OS, (C) MCO-CRC 5-Year RFS, (D) MCO-CRC 5-Year OS

Fig. 4

Kaplan-Meier curves illustrating stratification of 5-Year OS by the AI iTIL score in Stage II clinical high risk cases (A), and Stage II pMMR clinical high and low risk cases (B) in each of the validation cohorts. Significant stratification, particularly in the pMMR clinical high risk group, suggests the AI iTIL score could be used as an additional signal to identify patients with better/worse outcomes