Multicenter Study

. 2021 Mar 23;19(1):76.

doi: 10.1186/s12916-021-01942-5.

Accurate diagnosis of colorectal cancer based on histopathology images using artificial intelligence

K S Wang^#^{1

2}, G Yu^#³, C Xu^#⁴, X H Meng^#⁵, J Zhou^{1

2}, C Zheng^{1

2}, Z Deng^{1

2}, L Shang¹, R Liu¹, S Su¹, X Zhou¹, Q Li¹, J Li¹, J Wang¹, K Ma², J Qi², Z Hu², P Tang², J Deng⁶, X Qiu⁷, B Y Li⁷, W D Shen⁷, R P Quan⁷, J T Yang⁷, L Y Huang⁷, Y Xiao⁷, Z C Yang⁸, Z Li⁹, S C Wang¹⁰, H Ren^{11

12}, C Liang¹³, W Guo¹⁴, Y Li¹⁴, H Xiao¹⁵, Y Gu¹⁵, J P Yun¹⁶, D Huang¹⁷, Z Song¹⁸, X Fan¹⁹, L Chen²⁰, X Yan²¹, Z Li²², Z C Huang³, J Huang²³, J Luttrell²⁴, C Y Zhang²⁴, W Zhou²⁵, K Zhang²⁶, C Yi²⁷, C Wu²⁸, H Shen^{6

29}, Y P Wang^{6

30}, H M Xiao³¹, H W Deng^{32

33

34}

Affiliations

¹ Department of Pathology, Xiangya Hospital, Central South University, Changsha, 410078, Hunan, China.
² Department of Pathology, School of Basic Medical Science, Central South University, Changsha, 410013, Hunan, China.
³ Department of Biomedical Engineering, School of Basic Medical Science, Central South University, Changsha, 410013, Hunan, China.
⁴ Department of Biostatistics and Epidemiology, The University of Oklahoma Health Sciences Center, Oklahoma City, OK, 73104, USA.
⁵ Laboratory of Molecular and Statistical Genetics, College of Life Sciences, Hunan Normal University, Changsha, 410081, Hunan, China.
⁶ Department of Deming Department of Medicine, Tulane Center of Biomedical Informatics and Genomics, Tulane University School of Medicine, 1440 Canal Street, Suite 1610, New Orleans, LA, 70112, USA.
⁷ Centers of System Biology, Data Information and Reproductive Health, School of Basic Medical Science, School of Basic Medical Science, Central South University, Changsha, 410008, Hunan, China.
⁸ Department of Pharmacology, Xiangya School of Pharmaceutical Sciences, Central South University, Changsha, 410078, Hunan, China.
⁹ School of Life Sciences, Central South University, Changsha, 410013, Hunan, China.
¹⁰ College of Information Science and Engineering, Hunan Normal University, Changsha, 410081, Hunan, China.
¹¹ Department of Pathology, Gongli Hospital, Second Military Medical University, Shanghai, 200135, China.
¹² Department of Pathology, the Peace Hospital Affiliated to Changzhi Medical College, Changzhi, 046000, China.
¹³ Pathological Laboratory of Adicon Medical Laboratory Co., Ltd, Hangzhou, 310023, Zhejiang, China.
¹⁴ Department of Pathology, First Affiliated Hospital of Hunan Normal University, The People's Hospital of Hunan Province, Changsha, 410005, Hunan, China.
¹⁵ Department of Pathology, the Third Xiangya Hospital, Central South University, Changsha, 410013, Hunan, China.
¹⁶ Department of Pathology, Sun Yat-Sen University Cancer Center, Guangzhou, 510060, China.
¹⁷ Department of Pathology, Fudan University Shanghai Cancer Center, Shanghai, 200032, China.
¹⁸ Department of Pathology, Chinese PLA General Hospital, Beijing, 100853, China.
¹⁹ Department of Pathology, Nanjing Drum Tower Hospital, the Affiliated Hospital of Nanjing University Medical School, Nanjing, 210008, China.
²⁰ Department of Pathology, The first affiliated hospital, Air Force Medical University, Xi'an, 710032, China.
²¹ Institute of Pathology and southwest cancer center, Southwest Hospital, Third Military Medical University, Chongqing, 400038, China.
²² Department of Pathology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, 510080, China.
²³ Department of Anatomy and Neurobiology, School of Basic Medical Science, Central South University, Changsha, 410013, Hunan, China.
²⁴ School of Computing Sciences and Computer Engineering, University of Southern Mississippi, Hattiesburg, MS, 39406, USA.
²⁵ College of Computing, Michigan Technological University, Houghton, MI, 49931, USA.
²⁶ Department of Computer Science, Bioinformatics Facility of Xavier NIH RCMI Cancer Research Center, Xavier University of Louisiana, New Orleans, LA, 70125, USA.
²⁷ Department of Pathology, Ochsner Medical Center, New Orleans, LA, 70121, USA.
²⁸ Department of Statistics, Florida State University, Tallahassee, FL, 32306, USA.
²⁹ Division of Biomedical Informatics and Genomics, Deming Department of Medicine, Tulane University School of Medicine, New Orleans, LA, 70112, USA.
³⁰ Department of Biomedical Engineering, Tulane University, New Orleans, LA, 70118, USA.
³¹ Centers of System Biology, Data Information and Reproductive Health, School of Basic Medical Science, School of Basic Medical Science, Central South University, Changsha, 410008, Hunan, China. hmxiao@csu.edu.cn.
³² Department of Deming Department of Medicine, Tulane Center of Biomedical Informatics and Genomics, Tulane University School of Medicine, 1440 Canal Street, Suite 1610, New Orleans, LA, 70112, USA. hdeng2@tulane.edu.
³³ Centers of System Biology, Data Information and Reproductive Health, School of Basic Medical Science, School of Basic Medical Science, Central South University, Changsha, 410008, Hunan, China. hdeng2@tulane.edu.
³⁴ Division of Biomedical Informatics and Genomics, Deming Department of Medicine, Tulane University School of Medicine, New Orleans, LA, 70112, USA. hdeng2@tulane.edu.

^# Contributed equally.

PMID: 33752648
PMCID: PMC7986569
DOI: 10.1186/s12916-021-01942-5

Multicenter Study

Accurate diagnosis of colorectal cancer based on histopathology images using artificial intelligence

K S Wang et al. BMC Med. 2021.

. 2021 Mar 23;19(1):76.

doi: 10.1186/s12916-021-01942-5.

Authors

Affiliations

¹ Department of Pathology, Xiangya Hospital, Central South University, Changsha, 410078, Hunan, China.
² Department of Pathology, School of Basic Medical Science, Central South University, Changsha, 410013, Hunan, China.
³ Department of Biomedical Engineering, School of Basic Medical Science, Central South University, Changsha, 410013, Hunan, China.
⁴ Department of Biostatistics and Epidemiology, The University of Oklahoma Health Sciences Center, Oklahoma City, OK, 73104, USA.
⁵ Laboratory of Molecular and Statistical Genetics, College of Life Sciences, Hunan Normal University, Changsha, 410081, Hunan, China.
⁶ Department of Deming Department of Medicine, Tulane Center of Biomedical Informatics and Genomics, Tulane University School of Medicine, 1440 Canal Street, Suite 1610, New Orleans, LA, 70112, USA.
⁷ Centers of System Biology, Data Information and Reproductive Health, School of Basic Medical Science, School of Basic Medical Science, Central South University, Changsha, 410008, Hunan, China.
⁸ Department of Pharmacology, Xiangya School of Pharmaceutical Sciences, Central South University, Changsha, 410078, Hunan, China.
⁹ School of Life Sciences, Central South University, Changsha, 410013, Hunan, China.
¹⁰ College of Information Science and Engineering, Hunan Normal University, Changsha, 410081, Hunan, China.
¹¹ Department of Pathology, Gongli Hospital, Second Military Medical University, Shanghai, 200135, China.
¹² Department of Pathology, the Peace Hospital Affiliated to Changzhi Medical College, Changzhi, 046000, China.
¹³ Pathological Laboratory of Adicon Medical Laboratory Co., Ltd, Hangzhou, 310023, Zhejiang, China.
¹⁴ Department of Pathology, First Affiliated Hospital of Hunan Normal University, The People's Hospital of Hunan Province, Changsha, 410005, Hunan, China.
¹⁵ Department of Pathology, the Third Xiangya Hospital, Central South University, Changsha, 410013, Hunan, China.
¹⁶ Department of Pathology, Sun Yat-Sen University Cancer Center, Guangzhou, 510060, China.
¹⁷ Department of Pathology, Fudan University Shanghai Cancer Center, Shanghai, 200032, China.
¹⁸ Department of Pathology, Chinese PLA General Hospital, Beijing, 100853, China.
¹⁹ Department of Pathology, Nanjing Drum Tower Hospital, the Affiliated Hospital of Nanjing University Medical School, Nanjing, 210008, China.
²⁰ Department of Pathology, The first affiliated hospital, Air Force Medical University, Xi'an, 710032, China.
²¹ Institute of Pathology and southwest cancer center, Southwest Hospital, Third Military Medical University, Chongqing, 400038, China.
²² Department of Pathology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, 510080, China.
²³ Department of Anatomy and Neurobiology, School of Basic Medical Science, Central South University, Changsha, 410013, Hunan, China.
²⁴ School of Computing Sciences and Computer Engineering, University of Southern Mississippi, Hattiesburg, MS, 39406, USA.
²⁵ College of Computing, Michigan Technological University, Houghton, MI, 49931, USA.
²⁶ Department of Computer Science, Bioinformatics Facility of Xavier NIH RCMI Cancer Research Center, Xavier University of Louisiana, New Orleans, LA, 70125, USA.
²⁷ Department of Pathology, Ochsner Medical Center, New Orleans, LA, 70121, USA.
²⁸ Department of Statistics, Florida State University, Tallahassee, FL, 32306, USA.
²⁹ Division of Biomedical Informatics and Genomics, Deming Department of Medicine, Tulane University School of Medicine, New Orleans, LA, 70112, USA.
³⁰ Department of Biomedical Engineering, Tulane University, New Orleans, LA, 70118, USA.
³¹ Centers of System Biology, Data Information and Reproductive Health, School of Basic Medical Science, School of Basic Medical Science, Central South University, Changsha, 410008, Hunan, China. hmxiao@csu.edu.cn.
³² Department of Deming Department of Medicine, Tulane Center of Biomedical Informatics and Genomics, Tulane University School of Medicine, 1440 Canal Street, Suite 1610, New Orleans, LA, 70112, USA. hdeng2@tulane.edu.
³³ Centers of System Biology, Data Information and Reproductive Health, School of Basic Medical Science, School of Basic Medical Science, Central South University, Changsha, 410008, Hunan, China. hdeng2@tulane.edu.
³⁴ Division of Biomedical Informatics and Genomics, Deming Department of Medicine, Tulane University School of Medicine, New Orleans, LA, 70112, USA. hdeng2@tulane.edu.

^# Contributed equally.

PMID: 33752648
PMCID: PMC7986569
DOI: 10.1186/s12916-021-01942-5

Abstract

Background: Accurate and robust pathological image analysis for colorectal cancer (CRC) diagnosis is time-consuming and knowledge-intensive, but is essential for CRC patients' treatment. The current heavy workload of pathologists in clinics/hospitals may easily lead to unconscious misdiagnosis of CRC based on daily image analyses.

Methods: Based on a state-of-the-art transfer-learned deep convolutional neural network in artificial intelligence (AI), we proposed a novel patch aggregation strategy for clinic CRC diagnosis using weakly labeled pathological whole-slide image (WSI) patches. This approach was trained and validated using an unprecedented and enormously large number of 170,099 patches, > 14,680 WSIs, from > 9631 subjects that covered diverse and representative clinical cases from multi-independent-sources across China, the USA, and Germany.

Results: Our innovative AI tool consistently and nearly perfectly agreed with (average Kappa statistic 0.896) and even often better than most of the experienced expert pathologists when tested in diagnosing CRC WSIs from multicenters. The average area under the receiver operating characteristics curve (AUC) of AI was greater than that of the pathologists (0.988 vs 0.970) and achieved the best performance among the application of other AI methods to CRC diagnosis. Our AI-generated heatmap highlights the image regions of cancer tissue/cells.

Conclusions: This first-ever generalizable AI system can handle large amounts of WSIs consistently and robustly without potential bias due to fatigue commonly experienced by clinical pathologists. It will drastically alleviate the heavy clinical burden of daily pathology diagnosis and improve the treatment for CRC patients. This tool is generalizable to other cancer diagnosis based on image recognition.

Keywords: Cancer diagnosis; Colorectal cancer; Deep learning; Histopathology image.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no competing interests.

Figures

**Fig. 1**
Study pipeline and dataset usage

**Fig. 2**
Patient-level testing performance on twelve independent datasets from Dataset-C. Left: the radar map of the sensitivity, specificity, accuracy, and AUC in each dataset from Dataset-C. Right: the boxplot showing the distribution of sensitivity, specificity, accuracy, and AUC in datasets excluding XH and TCGA. The horizontal bar in the box indicates the median, while the cross indicates the mean. Circles represent data points

**Fig. 3**
ROC analysis of AI and pathologists in the Human-AI contest using Dataset-D. The blue line is the estimated ROC curve for AI. The colored triangles indicate the sensitivity and specificity achieved by the six pathologists

**Fig. 4**
Heatmap produced by AI. Top row: WSI from radical surgery (left) and colonoscopy (right); middle row: AI predicted heatmap corresponding to the first row, with white coloration indicating predicted cancer region; bottom row: heatmap overlaid on the corresponding WSI

See this image and copyright information in PMC

References

1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68(6):394–424. doi: 10.3322/caac.21492. - DOI - PubMed
1. Arnold M, Sierra MS, Laversanne M, Soerjomataram I, Jemal A, Bray F. Global patterns and trends in colorectal cancer incidence and mortality. Gut. 2017;66(4):683–691. doi: 10.1136/gutjnl-2015-310912. - DOI - PubMed
1. Komura D, Ishikawa S. Machine learning methods for histopathological image analysis. Comput Struct Biotechnol J. 2018;16:34–42. doi: 10.1016/j.csbj.2018.01.001. - DOI - PMC - PubMed
1. Maung R. Pathologists’ workload and patient safety. Diagnostic Histopathol. 2016;22(8):283–287. doi: 10.1016/j.mpdhp.2016.07.004. - DOI
1. Metter DM, Colgan TJ, Leung ST, Timmons CF, Park JY. Trends in the US and Canadian pathologist workforces from 2007 to 2017. JAMA Netw Open. 2019;2(5):e194337. doi: 10.1001/jamanetworkopen.2019.4337. - DOI - PMC - PubMed

Publication types

Actions
Actions
Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources
Other Literature Sources
- scite Smart Citations
Medical
- MedlinePlus Health Information

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Accurate diagnosis of colorectal cancer based on histopathology images using artificial intelligence

Affiliations

Accurate diagnosis of colorectal cancer based on histopathology images using artificial intelligence

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources

Medical