Predicting microsatellite instability and key biomarkers in colorectal cancer from H&E-stained images: achieving state-of-the-art predictive performance with fewer data using Swin Transformer

Abstract

Many artificial intelligence models have been developed to predict clinically relevant biomarkers for colorectal cancer (CRC), including microsatellite instability (MSI). However, existing deep learning networks require large training datasets, which are often hard to obtain. In this study, based on the latest Hierarchical Vision Transformer using Shifted Windows (Swin Transformer [Swin-T]), we developed an efficient workflow to predict biomarkers in CRC (MSI, hypermutation, chromosomal instability, CpG island methylator phenotype, and BRAF and TP53 mutation) that required relatively small datasets. Our Swin-T workflow substantially achieved the state-of-the-art (SOTA) predictive performance in an intra-study cross-validation experiment on the Cancer Genome Atlas colon and rectal cancer dataset (TCGA-CRC-DX). It also demonstrated excellent generalizability in cross-study external validation and delivered a SOTA area under the receiver operating characteristic curve (AUROC) of 0.90 for MSI, using the Molecular and Cellular Oncology dataset for training (N = 1,065) and the TCGA-CRC-DX (N = 462) for testing. A similar performance (AUROC = 0.91) was reported in a recent study, using ~8,000 training samples (ResNet18) on the same testing dataset. Swin-T was extremely efficient when using small training datasets and exhibited robust predictive performance with 200-500 training samples. Our findings indicate that Swin-T could be 5-10 times more efficient than existing algorithms for MSI prediction based on ResNet18 and ShuffleNet. Furthermore, the Swin-T models demonstrated their capability in accurately predicting MSI and BRAF mutation status, which could exclude and therefore reduce samples before subsequent standard testing in a cascading diagnostic workflow, in turn reducing turnaround time and costs.

Keywords: Swin Transformer; biomarkers; colorectal cancer; deep learning; digital pathology.

Figure 1

The workflow of the data preprocessing and the training process of the DL model. (A) Tiles images of NCT‐CRC‐HE‐100K are downloaded from the publicly available website (https://zenodo.org/record/1214456) to pre‐train a tissue classifier based on Swin‐T. The classifier has excellent performance of classifying tissues (overall accuracy = 96.3%) and detecting tumor tiles (accuracy = 98%) in an external dataset: CRC‐VAL‐HE‐7K. (B) WSIs in the SVS format of the MCO dataset and TCGA dataset are preprocessed to tessellate into nonoverlapping tiles with a size of 512 × 512 pixels. These tiles are then resized to the smaller 224 × 224 pixels tiles and color normalized. The tumor tiles are selected. (C) For each patient, up to 500 tiles are randomly sampled for subsequent experiments. The pre‐trained tissue classifier model in (A) is then fine‐tuned to predict biomarker status of each tile. The probability values of the tiles are averaged to derive the prediction at the patient level. The performance of the models is evaluated in two separate experiments: an intra‐cohort four‐fold cross‐validation and an inter‐cohort external validation.

Figure 2

Predictive performance of four‐fold cross‐validation of Swin‐T based prediction of colorectal cancer biomarkers in the TCGA‐CRC‐DX cohort. AUROC plots for prediction of hypermutation (HM), MSI, CING, CIMP, BRAF mutation status, and TP53 mutation status. The true positive rate represents sensitivity and the false positive rate represents 1 − specificity. The red shaded areas represent the SD. The value in the lower right of each plot represents mean AUROC ± SD.

Figure 3

Predictive performance of intra‐cohort four‐fold cross‐validation in the MCO cohort and inter‐cohort external validation in the TCGA‐CRC‐DX cohort: MSI, BRAF mutation status (BRAF), CIMP. (A) AUROC plots for four‐fold cross‐validation in MCO cohort. The red shaded areas represent the SD. The value in the lower right of each plot represents mean AUROC ± SD. (B) AUROC plots for inter‐cohort external validation in TCGA‐CRC‐DX cohort. The red shaded areas represent the 95% confidence interval (CI), calculated by 1,000× bootstrap. The values in the lower right of each plot represent mean AUROC (95% CI).

Figure 4

Test statistics for the pre‐screening tool. Test performance of MSI status, BRAF mutation, and CIMP status in the TCGA‐CRC‐DX cohorts displayed as patients classified true/false positive/negative by the Swin‐T model based on 95% sensitivity threshold and fixed thresholds (0.25, 0.5, and 0.75).

Figure 5

Visualization of the reader study of representative TP (MSI) and TN (MSS) cases. (A–D) Tissue slides for TP cases and signature pathological features identified by the pathologist. (E) Tissue slides for TN cases and signature pathological features identified by the pathologist.

Figure 6

Visualization of the reader study of representative misclassified cases. (A–C) Tissue slides for FP cases and potential confounding pathological features and misclassification reasons identified by the pathologist. (D–F) Tissue slides for FN cases and potential confounding pathological features and misclassification reasons identified by the pathologist.