LINNDA: Lymphoma identification through neural network detection aid

Abstract

Preoperative differentiation between primary central nervous system lymphoma (PCNSL) and glioblastoma (GBM) is crucial for appropriate management and surgical planning. This study aims to evaluate the diagnostic performance of the AI-assisted workflow, LINNDA (lymphoma identification through neural network detection aid), in comparison to that of human raters. In total, ten clinicians independently reviewed 46 cases of GBM and PCNSL. The LINNDA workflow evaluated all 1,470 possible pairwise combinations. For each pair, whenever two clinicians disagreed, a DenseNet169 neural network was explicitly integrated as a third independent diagnostic opinion ("tie-breaker"). Integrating the AI-generated predictions improved overall accuracy to 89.9%, exceeding the expert consensus. We further established the superiority of our approach over a third human rater in another 5,108 possible combinatory scenarios. LINNDA has a negative predictive value of 97% for ruling out the diagnosis of PCNSL, providing a sound basis for clinical decision-making.

Keywords: Artificial intelligence; Cancer.

Graphical abstract

Figure 1

Combination of manual and automated tumor classification on dataset 1 Top: raters (R1-10) showed heterogeneous performances regarding tumor classification of dataset 1 (pie charts and Table 1). The correlation, assessed by Spearman’s r, between cases diagnosed correctly and incorrectly by raters is color-coded by the connecting lines. No significant difference was found when comparing the performance of different raters based on the four primary sequences in dataset 1 (see text), either for all cases or when comparing PCNSL and GBM separately (A), two-way ANOVA with multiple comparisons, p > 0.5. Likewise, no significant differences in diagnosis accuracy improvement following access to all sequences were found between groups (B). Correlations between rater diagnoses were homogenously high among clinicians, irrespective of their specialization, but significantly lower between clinicians and the CNN (C), unpaired t test (full data in Table S2). All data are represented as mean ± SD.

Figure 2

Superior diagnostic accuracy through combining human and artificial intelligence (A) Sankey chart depicting the diagnostic workflow; of the 1,470 possible combinations, 268 cases were diagnosed differently by two raters. In most cases where both raters agreed, the diagnosis was correct. In cases with dissenting clinical ratings, the CNN was tasked with diagnosing the lesions, yielding a total accuracy of over 89.9% (Table 2). (B) LINNDA significantly outperformed both the performance of the CNN alone, the joint diagnosis of two human raters, as well as the joint diagnosis of three human raters when comparing the ratios of overall predictions (n = 1,470) (∗p < 0.01, Fisher’s exact test). (C) A significant correlation was found between reported confidence in diagnosis and correct (p < 0.0001 for the four main sequences; p = 0.0002 for all sequences, logistic regression). (D) Diagnostic accuracy improved when raters were given access to all imaging sequences, an effect mainly carried by the combination of surgeons with radiologists (∗∗∗∗p < 0.0001, ∗∗p < 0.001; Fisher’s exact test).

Figure 3

Addressing the domain shift (A) No differences in segmented tumor volumes were detected between datasets 1 and 2 described here (p > 0.5; two-way ANOVA). All data are represented as mean ± SD; likewise, no differences in the geometric tumor location were detected (p > 0.5; two-way ANOVA). (B and C) Both datasets showed no significant differences regarding the scanners used (C). See also Table S4. D1–D3 shows example MR images of a PCNSL patient included in the study. E1–E3 are similar examples of a GBM patient.