Assessment of individual tumor buds using keratin immunohistochemistry: moderate interobserver agreement suggests a role for machine learning

Abstract

Tumor budding is a promising and cost-effective biomarker with strong prognostic value in colorectal cancer. However, challenges related to interobserver variability persist. Such variability may be reduced by immunohistochemistry and computer-aided tumor bud selection. Development of computer algorithms for this purpose requires unequivocal examples of individual tumor buds. As such, we undertook a large-scale, international, and digital observer study on individual tumor bud assessment. From a pool of 46 colorectal cancer cases with tumor budding, 3000 tumor bud candidates were selected, largely based on digital image analysis algorithms. For each candidate bud, an image patch (size 256 × 256 µm) was extracted from a pan cytokeratin-stained whole-slide image. Members of an International Tumor Budding Consortium (n = 7) were asked to categorize each candidate as either (1) tumor bud, (2) poorly differentiated cluster, or (3) neither, based on current definitions. Agreement was assessed with Cohen's and Fleiss Kappa statistics. Fleiss Kappa showed moderate overall agreement between observers (0.42 and 0.51), while Cohen's Kappas ranged from 0.25 to 0.63. Complete agreement by all seven observers was present for only 34% of the 3000 tumor bud candidates, while 59% of the candidates were agreed on by at least five of the seven observers. Despite reports of moderate-to-substantial agreement with respect to tumor budding grade, agreement with respect to individual pan cytokeratin-stained tumor buds is moderate at most. A machine learning approach may prove especially useful for a more robust assessment of individual tumor buds.

Fig. 1. Example of TB-candidate in IHC and re-stained H&E.

Example of TB-candidate in a immunohistochemistry, and b restained H&E.

Fig. 2. Examples of TB-candidates with manual scores.

Examples of TB-candidates a, b uniform selected as bud, c, d majority vote bud, e, f uniform PDC, g, h majority vote PDC, i, j uniform neither, k, l majority vote neither. m–p no agreement was reached. Legend of colors: blue—bud, orange—PDC, gray—neither.

Fig. 3

TB, PDC and neither scores in the 2 × 1500 immunohistochemistry dataset per observer.

Fig. 4

One-versus-one Cohen Kappa scores per observer of left group 1 and right group 2.

Fig. 5

Cohen Kappa scores per observer for the 150 H&E cases.

Fig. 6

TB, PDC, and neither scores in the 150 H&E dataset per observer with the addition of scores of the identical objects in immunohistochemistry staining.

Fig. 7. Score shifted examples of restained TB-candidates.

a, b Shift in the agreement from uniform bud vote in immunohistochemistry to no agreement in H&E, c shift from majority vote on tumor bud in immunohistochemistry to uniform neither assessment in H&E.