Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jul 13;13(1):11314.
doi: 10.1038/s41598-023-38109-6.

Region of interest (ROI) selection using vision transformer for automatic analysis using whole slide images

Md Shakhawat Hossain  1   2 , Galib Muhammad Shahriar  3 , M M Mahbubul Syeed #  4   3 , Mohammad Faisal Uddin #  4   3 , Mahady Hasan #  4   3 , Shingla Shivam #  5 , Suresh Advani #  5
Affiliations

Region of interest (ROI) selection using vision transformer for automatic analysis using whole slide images

Md Shakhawat Hossain et al. Sci Rep. .

Abstract

Selecting regions of interest (ROI) is a common step in medical image analysis across all imaging modalities. An ROI is a subset of an image appropriate for the intended analysis and identified manually by experts. In modern pathology, the analysis involves processing multidimensional and high resolution whole slide image (WSI) tiles automatically with an overwhelming quantity of structural and functional information. Despite recent improvements in computing capacity, analyzing such a plethora of data is challenging but vital to accurate analysis. Automatic ROI detection can significantly reduce the number of pixels to be processed, speed the analysis, improve accuracy and reduce dependency on pathologists. In this paper, we present an ROI detection method for WSI and demonstrated it for human epidermal growth factor receptor 2 (HER2) grading for breast cancer patients. Existing HER2 grading relies on manual ROI selection, which is tedious, time-consuming and suffers from inter-observer and intra-observer variability. This study found that the HER2 grade changes with ROI selection. We proposed an ROI detection method using Vision Transformer and investigated the role of image magnification for ROI detection. This method yielded an accuracy of 99% using 20 × WSI and 97% using 10 × WSI for the ROI detection. In the demonstration, the proposed method increased the diagnostic agreement to 99.3% with the clinical scores and reduced the time to 15 seconds for automated HER2 grading.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Figure 1
Figure 1
Multi-layer pyramid model of WSI.
Figure 2
Figure 2
Human epidermal growth factor receptor 2 (HER2) status changes depending on the selection of region regardless the test methods.
Figure 3
Figure 3
Visualization of ROI detection result produced by proposed method where green, yellow and purple colored-box indicates representative, artifact-affected and poor quality ROIs.
Figure 4
Figure 4
Overlap of ROIs for proposed method and pathologists.
Figure 5
Figure 5
Box-plot of machine learning models for 10 × and 20 × images.
Figure 6
Figure 6
ROC for 5-fold cross validation for 10 × images.
Figure 7
Figure 7
ROC for 5-fold cross validation for 20 × images.
Figure 8
Figure 8
Proposed ViT architecture for ROI detection from WSI.
Figure 9
Figure 9
Example of nine different approaches to gradually reduce the neurons to fit two output neurons in the final layer.
Figure 10
Figure 10
Design of automated WSI analysis using proposed method.
See this image and copyright information in PMC

References

    1. Hossain, M. S. et al.. Tissue artifact segmentation and severity assessment for automatic analysis using wsi. IEEE Access (2023).
    1. Caccomo S. Fda Allows Marketing of First Whole Slide Imaging System for Digital Pathology. US Department of Health and Human Services; 2017.
    1. Snead DR, et al. Validation of digital pathology imaging for primary histopathological diagnosis. Histopathology. 2016;68:1063–1072. doi: 10.1111/his.12879. - DOI - PubMed
    1. Pantanowitz L, et al. Validating whole slide imaging for diagnostic purposes in pathology: Guideline from the college of American pathologists pathology and laboratory quality center. Arch. Pathol. Lab. Med. 2013;137:1710–1722. doi: 10.5858/arpa.2013-0093-CP. - DOI - PMC - PubMed
    1. Hossain MS, et al. Automatic quantification of her2 gene amplification in invasive breast cancer from chromogenic in situ hybridization whole slide images. J. Med. Imaging. 2019;6:047501. doi: 10.1117/1.JMI.6.4.047501. - DOI - PMC - PubMed

Publication types