Fully Automated Artificial Intelligence Solution for Human Epidermal Growth Factor Receptor 2 Immunohistochemistry Scoring in Breast Cancer: A Multireader Study

Abstract

Purpose: The proven efficacy of human epidermal growth factor receptor 2 (HER2) antibody-drug conjugate therapy for treating HER2-low breast cancers necessitates more accurate and reproducible HER2 immunohistochemistry (IHC) scoring. We aimed to validate performance and utility of a fully automated artificial intelligence (AI) solution for interpreting HER2 IHC in breast carcinoma.

Materials and methods: A two-arm multireader study of 120 HER2 IHC whole-slide images from four sites assessed HER2 scoring by four surgical pathologists without and with the aid of an AI HER2 solution. Both arms were compared with high-confidence ground truth (GT) established by agreement of at least four of five breast pathology subspecialists according to ASCO/College of American Pathologists (CAP) 2018/2023 guidelines.

Results: The mean interobserver agreement among GT pathologists across all HER2 scores was 72.4% (N = 120). The AI solution demonstrated high accuracy for HER2 scoring, with 92.1% agreement on slides with high confidence GT (n = 92). The use of the AI tool led to improved performance by readers, interobserver agreement increased from 75.0% for digital manual read to 83.7% for AI-assisted review, and scoring accuracy improved from 85.3% to 88.0%. For the distinction of HER2 0 from 1+ cases (n = 58), pathologists supported by AI showed significantly higher interobserver agreement (69.8% without AI v 87.4% with AI) and accuracy (81.9% without AI v 88.8% with AI).

Conclusion: This study demonstrated utility of a fully automated AI solution to aid in scoring HER2 IHC accurately according to ASCO/CAP 2018/2023 guidelines. Pathologists supported by AI showed improvements in HER2 IHC scoring consistency and accuracy, especially for distinguishing HER2 0 from 1+ cases. This AI solution could be used by pathologists as a decision support tool for enhancing reproducibility and consistency of HER2 scoring and particularly for identifying HER2-low breast cancers.

FIG 1.

Overview of the AI algorithm. The HER2 IHC whole-slide images are uploaded to the system, tissue is then detected using the tissue detection algorithm (step 1), and on-slide control (if present) is located and excluded (step 2). Then, the region classification AI model identifies the invasive cancer (step 3), and within it, tumor cells are detected and classified using the cell detection AI model (step 4). Finally, a slide-level HER2 score is calculated, and visualization of the previous steps is prepared and displayed to the user in the Galen slide viewer (steps 5 and 6). AI, artificial intelligence; AOI, area of interest; CAP, College of American Pathologists; HER2, human epidermal growth factor receptor 2; IHC, immunohistochemistry.

FIG 2.

GT analysis. (A) Slide distribution per HER2 score as determined by the majority agreement of five breast pathology subspecialists performing GT (N = 120). (B) HER2 scores for slides with high-confidence GT (n = 92); (C) HER2 scores for slides with low-confidence GT (majority of 3 of 5; n = 27)—mainly 0/1+ (ie, slides that were given a score of 0 by three breast experts and 1+ by two breast experts or vice versa) and 1+/2+ slides. GT, ground truth; HER2, human epidermal growth factor receptor 2.

FIG 3.

Percentages of HER2 scores by expert GT and reader pathologists. Percentages per HER2 score for the entire study cohort (N = 120 slides) for (A) GT expert breast pathologists, (B) reader pathologists without the AI algorithm (n = 119, one slide with no GT was excluded) average pair agreement, and (C) reader pathologists with AI (n = 119). AI, artificial intelligence; GT, ground truth; HER2, human epidermal growth factor receptor 2.

FIG 4.

Reader pathologists' performance for all HER2 scores. (A) Interobserver agreement for pathologists, with and without the AI algorithm, for all slides with GT (n = 119). (B) Interobserver agreement for pathologists with and without AI for high-confidence GT slides (n = 92). (C) Interobserver agreement for pathologists with and without AI for HER2 0 and 1+ slides with high-confidence GT (n = 58). (D) Reader pathologists' accuracy (ie, agreement with GT) with and without AI for slides with high-confidence GT (n = 92). (E) Reader pathologists' accuracy with and without AI for HER2 0 and 1+ slides with high-confidence GT (n = 58); Average percentage agreement and 95% CI are presented as bar with error graphs. AI, artificial intelligence; GT, ground truth; HER2, human epidermal growth factor receptor 2; STD, standard deviation.

FIG 5.

Examples of the AI algorithm's effect on pathologists' review. Shown are slides with borderline HER2 IHC scores of 0/1+ (A) without AI and (B) with AI (total percentage of stained cells is 3.6% per the AI), both at 18× magnification (0.56 μm/pixel) with higher magnification of 40× (0.25 μm/pixel) insert so that the cell overlays provided by the AI solution can be visualized, and slides scored HER2 1+/2+ (C) without AI and (D) with AI (percentage of faint stained cells is 23.2% and moderate complete and incomplete 2% per the AI) at 18× (0.56 μm/pixel) with higher magnification of 40× (0.25 μm/pixel) insert; (B and D) red contour line marks the invasive tumor area detected by the AI, with (D) DCIS correctly excluded by the AI from the area of interest; (B) HER2 score and detected % for different cell staining patterns are provided in the AI slide report, with cell overlay (small colored circles) indicating different HER2 cell staining patterns detected by the AI (eg, empty blue—not stained, green—faintly stained, etc). AI, artificial intelligence; DCIS, ductal carcinoma in situ; HER2, human epidermal growth factor receptor 2; IHC, immunohistochemistry.