Deep learning enabled liquid-based cytology model for cervical precancer and cancer detection

Abstract

Deep learning (DL) enabled liquid-based cytology has potential for cervical cancer screening or triage. Here, we develop a DL model using whole cytology slides from 17,397 women and test it on 10,826 additional cases through a three-stage process. The DL model achieves robust performance across nine hospitals. In a multi-reader, multi-case study, it outperforms cytopathologists' sensitivity by 9%. Reading time significantly decreases with DL assistance (218s vs 30s; p < 0.0001). In community-based organized screening, the DL model's sensitivity matches that of senior cytopathologists (0.878 vs 0.854; p > 0.999), yet it has reduced specificity (0.831 vs 0.901; p < 0.0001). Notably, hospital-based opportunistic screening shows that junior cytopathologists with DL assistance significantly improve both their sensitivity and specificity (0.857 vs 0.657, 0.840 vs 0.737; both p < 0.0001). When triaging human papillomavirus-positive cases, DL assistance exhibits better performance than junior cytopathologists alone. These findings support using the DL model as an assistance tool in cervical screening and case triage.

Fig. 1. Study profile.

This DL model was trained to classify LBC digital slides as positive or negative cases using data from two independent pathology archives. By histological reference standards, the DL model was then tested in a stepwise validation study which had three stages (test sets A–D). First, a multi-institutional validation study was performed to assess the generalizability across nine hospitals. Second, diagnostic performance and efficiency of cytopathologists before and after DL assistance was assessed in a multi-reader multi-case study. Third, diagnostic performance and referral efficiency between senior cytopathologists and DL alone were evaluated using a cloud platform in a community-based organized screening population. Diagnostic performance and referral efficiency of junior cytopathologists before and after DL assistance was assessed locally in a hospital-based opportunistic screening population. DL Deep learning, LBC Liquid-based cytology.

Fig. 2. Comparison of diagnostic performance of DL model and cytopathologists in the multi-institutional validation study and multi-reader multi-case study for CIN2+ detection.

Multi-institutional validation study (A–C): A ROC curves for diagnostic performance of DL alone across nine different hospitals. B Diagnostic performance of DL model compared with each junior cytopathologist. Blue dots indicate diagnostic sensitivities and specificities for individual junior cytopathologists. Blue rhombus indicates the average sensitivity and specificity for all junior cytopathologists. C Diagnostic performance of DL model compared with each senior cytopathologist. Orange dots indicate sensitivities and specificities for individual senior cytopathologists. Orange rhombus indicates the average sensitivity and specificity of all senior cytopathologists. Multi-reader multi-case study (D–F): D Diagnostic sensitivities and specificities of individual junior and senior cytopathologists before and after DL assistance. Blue dots indicate sensitivities and specificities of individual cytopathologists without DL assistance. Orange triangles indicate sensitivities and specificities of individual cytopathologists with the use of DL assistance. E ROC curves for diagnostic performance of DL alone, junior and senior cytopathologists with and without DL assistance. Blue and orange dots indicate sensitivities and specificities of individual junior cytopathologists without and with DL assistance, respectively. Blue and orange rhombi indicate sensitivities and specificities of individual senior cytopathologists without and with DL assistance, respectively. Blue and orange rhombi indicate average sensitivities and specificities of all junior cytopathologists without and with DL assistance, respectively. Blue and orange stars indicate average sensitivities and specificities of all senior cytopathologists without and with DL assistance, respectively. F The average reading time is measured for all cytopathologists (n = 28), junior cytopathologists (n = 16), and senior cytopathologists (n = 12) with and without DL assistance. The data represent independent assessments by each group, with biological replicates defined as separate assessments by different cytopathologists with and without DL assistance. The upper and lower bounds of the box represent the 75th percentile (Q3) and 25th (Q1) percentile, respectively. The line within the box indicates the median. The rhombus outside of whiskers refer to outliers. The time of review per case is described as the median and interquartile range (IQR). CIN2+ Cervical intraepithelial neoplasia grade 2 or worse, AUC Area under the receiver operating characteristic curves, DL Deep learning, APH Anhui Provincial Hospital, GZPH Guangxi Zhuang Autonomous Region People’s Hospital, HHMU The First Affiliated Hospital of Hainan Medical University, WCSUH West China Second University Hospital, GHPLA The 7th Medical Center, General Hospital of PLA, SMCHH Shenzhen Maternity and Child Healthcare Hospital, NWCH Northwest Women’s and Children’s Hospital, XH Xijing Hospital, ZCH Zhejiang Cancer Hospital, IQR Interquartile Range.

Fig. 3. Comparison of diagnostic performance and efficiency of DL model and cytopathologists in two application validation studies for CIN2+ detection.

The application validation study 1 was conducted in a community-based organized screening population (A–C): A ROC curves for diagnostic performance of DL alone and senior cytopathologists. Blue triangles indicate the average sensitivities and specificities of senior cytopathologists. B Diagnostic accuracy, sensitivity and specificity between DL alone and the average of senior cytopathologists. c Referral efficiency includes colposcopy referrals and NNR between DL alone and the average of senior cytopathologists. The application validation study 2 was conducted in a hospital-based opportunistic screening population (D–F): D ROC curves for diagnostic performance of DL alone and junior cytopathologists with and without DL assistance. Blue and orange triangles indicate the average sensitivities and specificities of junior cytopathologists with and without DL assistance, respectively. E Diagnostic accuracy, sensitivity and specificity, for DL alone and junior cytopathologists with and without DL assistance. F Referral efficiency including colposcopy referrals and NNR for DL alone and junior cytopathologists with and without DL assistance. CIN2+ Cervical intraepithelial neoplasia grade 2 or worse, AUC Area under the receiver operating characteristic curves, DL Deep learning, NNR Number of colposcopies required to yield one CIN2+.