Analytical validation of CanAssist-Breast: an immunohistochemistry based prognostic test for hormone receptor positive breast cancer patients

Abstract

Background: CanAssist-Breast is an immunohistochemistry based test that predicts risk of distant recurrence in early-stage hormone receptor positive breast cancer patients within first five years of diagnosis. Immunohistochemistry gradings for 5 biomarkers (CD44, ABCC4, ABCC11, N-Cadherin and pan-Cadherins) and 3 clinical parameters (tumor size, tumor grade and node status) of 298 patient cohort were used to develop a machine learning based statistical algorithm. The algorithm generates a risk score based on which patients are stratified into two groups, low- or high-risk for recurrence. The aim of the current study is to demonstrate the analytical performance with respect to repeatability and reproducibility of CanAssist-Breast.

Methods: All potential sources of variation in CanAssist-Breast testing involving operator, run and observer that could affect the immunohistochemistry performance were tested using appropriate statistical analysis methods for each of the CanAssist-Breast biomarkers using a total 309 samples. The cumulative effect of these variations in the immunohistochemistry gradings on the generation of CanAssist-Breast risk score and risk category were also evaluated. Intra-class Correlation Coefficient, Bland Altman plots and pair-wise agreement were performed to establish concordance on IHC gradings, risk score and risk categorization respectively.

Results: CanAssist-Breast test exhibited high levels of concordance on immunohistochemistry gradings for all biomarkers with Intra-class Correlation Coefficient of ≥0.75 across all reproducibility and repeatability experiments. Bland-Altman plots demonstrated that agreement on risk scores between the comparators was within acceptable limits. We also observed > 90% agreement on risk categorization (low- or high-risk) across all variables tested.

Conclusions: The extensive analytical validation data for the CanAssist-Breast test, evaluating immunohistochemistry performance, risk score generation and risk categorization showed excellent agreement across variables, demonstrating that the test is robust.

Keywords: Analytical validation; CanAssist-breast; Immunohistochemistry; Repeatability; Reproducibility.

Fig. 1

CanAssist-Breast (CAB) test work flow: The flow chart depicts various steps involved in the whole process of testing a tumor sample for CanAssist-Beast. The FFPE tumor block undergoes a quality check by H&E staining. IHC is performed for all 5 CAB biomarkers. Trained observer (pathologist) grades the slides. The statistical algorithm generates risk-score using gradings of biomarkers and clinical parameters. Based on risk score, risk category is assigned and finally report is generated

Fig. 2

Overview of Precision study for analytical validation of CAB test: All potential sources of variation in the CAB test have been shown along with the methodology used to analyze. Sample size for each of the variable tested is given in brackets

Fig. 3

Design of Precision experiments Potential sources of variations are tested as detailed above (a-e) with sample size of 309, samples ranged from 41 to 257 for various experiments performed

Fig. 4

Spread of Biomarker staining Dot plot shows the spread of the staining percentage for the five biomarkers

Fig. 5

Distribution of CAB risk scores across all the tumor samples used for precision experiments: Dot plot shows the distribution of risk scores for all the tumor samples (n = 309) used for the precision experiments.

Fig. 6

Bland Altman Plots showing correlation for risk scores between variables: Bland Altman plot for risk score between average scores of three observers versus observer 1 (a),observer 2 (b), observer 3 (c); average scores of three gradings of single observer versus grading 1 (d), grading 2 (e), grading 3 (f); average scores of three gradings of 3 operators versus operator 1 (g), operator 2 (h), operator 3 (i); average scores of three runs performed by an operators versus run 1 (j), run 2 (k), run 3 (l); average scores of three readings performed by an operator in a run versus reading 1 (m), reading 2 (n), reading 3 (o)

Fig. 7

Bland Altman Plots for risk scores between variables around CAB clinical decision point, 13–18: Bland Altman plots for risk scores between average scores of three observers versus observer 1 (a), observer 2 (b), observer 3 (c); average scores of three gradings of single observer versus grading 1 (d), grading 2 (e), grading 3 (f)