Multi-assistant methods improve stromal tumor-infiltrating lymphocytes (sTILs) assessment in breast cancer: results of multi-institutional ring studies

Abstract

Background: Stromal tumor-infiltrating lymphocytes (sTILs) have significant prognostic value for breast cancer patients, but its accurate assessment can be very challenging. We comprehensively studied the pitfalls faced by pathologists with different levels of professional experience, and explored clinical applicability of reference cards (RCs)- and artificial intelligence (AI)-assisted methods in assessing sTILs.

Materials and methods: Three rounds of ring studies (RSs) involving 12 pathologists from four hospitals were conducted. AI algorithms based on the field of view (FOV) and whole section were proposed to create RCs and to compute whole-slide image interpretations, respectively. Stromal regions identified and the associated sTIL scores by the AI method were provided to the pathologists as references. Fifty cases of surgical resections were used for interobserver concordance analysis in RS1. A total of 200 FOVs with challenge factors were assessed in RS2 for accuracy of the RC-assisted and AI-assisted methods, while 167 cases were used to validate their clinical performance in RS3.

Results: With the assistance of RCs, the intraclass correlation coefficient (ICC) in RS1 increased significantly to 0.834 [95% confidence interval (CI) 0.772-0.889]. The largest enhancement in ICC, from moderate (ICC: 0.592; 95% CI 0.499-0.677) to good (ICC: 0.808; 95% CI 0.746-0.857) was observed for heterogeneity. Accuracy evaluation showed significant grade improvement for heterogeneity and stromal factor FOVs among senior, intermediate, and junior groups. The ICC of heterogeneity and stromal factor analysis by the AI-assisted method achieved a level comparable to that of the senior group with RC assistance. The area under the receiver operating characteristic (ROC) curve, denoted as AUC, for AI-assisted sTIL scores in predicting pathological complete response after neoadjuvant therapy was 0.937, which was superior to visual assessment with an AUC of 0.775.

Conclusion: RC- and AI-assisted technology can reduce the uncertainty of interpretation caused by heterogeneous distribution.

Keywords: AI; breast cancer; concordance; reference card; stromal TILs.

Figure 1

Pitfalls introduced in assessing sTILs in breast cancer. (A-B) Marked heterogeneity in sTILs density within the tumor. (B) Lymphocytic infiltrates of varying densities separated by collagenous stroma. (C-I) Challenges associated with delineating the stromal regions including limited stromal areas within tumor (C), necrosis (D), adipose tissue (E), peritumoral retraction clefting (F), histiocyte response (G), cholesterol deposition (H), and lipofuscin deposition (I). A: H&E staining, 4×; B-I: H&E staining, 10×. H&E, hematoxylin–eosin; sTILs, stromal tumor-infiltrating lymphocytes.

Figure 2

System diagram of the AI-W algorithm. The AI-F method was integrated as a building block to analyze all image FOVs in the WSI. The processing of FOVs with both high and low expressions of sTILs was exemplified, with intermediate cell-level and region-level results demonstrated in a pipeline. All of the mononuclear inflammatory cell (MIC)-occupied pixels in the WSI constituted the numerator of sTILs, while the total number of tumor-associated stroma pixels was the denominator. AI-F, artificial intelligence-field of view; AI-W, artificial intelligence; FOVs, fields of view; sTILs, stromal tumor-infiltrating lymphocytes; WSI, whole-slide image.

Figure 3

Concordance results in the hierarchical analysis of challenge factors. Inter-pathologist concordance in VA (A) and reference card-assisted interpretation (B). Heter: heterogeneity; OC, overall challenge factors (heterogeneity and stromal factors); Stro, all challenges associated with delineating the stromal regions.

Figure 4

Accuracy evaluation results. (A) ICC values of individual pathologists’ assessments and the gold standard for heterogeneity and stromal factor FOVs, respectively: traditional visual assessment (VA) method (red dashed lines) and the reference card-assisted (RC-AS) method (blue dashed lines). (B) ICC values and 95% CI for RC-AS and AI-assisted methods, with concordance ranges (excellent, good, moderate, and poor) represented as bands of varying grayscale intensities. AI-AS, artificial intelligence-assisted; CI, confidence interval; FOVs, fields of view; ICC, intraclass correlation coefficient.

Figure 5

Comparison of accuracy evaluation among different groups. Accuracy evaluation using ICC for pathologist groups with varying experience levels: comparison between VA (green) and RC-assisted method (red) concerning heterogeneity (A) and stromal factors (B). ICC, intraclass correlation coefficient; RC, reference card; VA, visual assessment.