Unraveling tumor specific neoantigen immunogenicity prediction: a comprehensive analysis

Abstract

Introduction: Identification of tumor specific neoantigen (TSN) immunogenicity is crucial to develop peptide/mRNA based anti-tumoral vaccines and/or adoptive T-cell immunotherapies; thus, accurate in-silico classification/prioritization proves critical for cost-effective clinical applications. Several methods were proposed as TSNs immunogenicity predictors; however, comprehensive performance comparison is still lacking due to the absence of well documented and adequate TSN databases.

Methods: Here, by developing a new curated database having 199 TSNs with experimentally-validated MHC-I presentation and positive/negative immune response (ITSNdb), sixteen metrics were evaluated as immunogenicity predictors. In addition, by using a dataset emulating patient derived TSNs and immunotherapy cohorts containing predicted TSNs for tumor neoantigen burden (TNB) with outcome association, the metrics were evaluated as TSNs prioritizers and as immunotherapy response biomarkers.

Results: Our results show high performance variability among methods, highlighting the need for substantial improvement. Deep learning predictors were top ranked on ITSNdb but show discrepancy on validation databases. In overall, current predicted TNB did not outperform existing biomarkers.

Conclusion: Recommendations for their clinical application and the ITSNdb are presented to promote development and comparison of computational TSNs immunogenicity predictors.

Keywords: cancer immunology; immunogenic neoantigen database; immunoinformatic; immunotherapy; neopeptide.

Figure 1

Benchmark workflow scheme. (A) The algorithm of inclusion criteria for the tumor specific neoantigens (immunogenic not immunogenic) into the ITSNdb. (B) The seven predictive software (three neoantigen binder predictors: netMHCpan, MHCFlurry and MixMHCPred, and four neoantigen immunogenicity predictors: CIImm, Prime, DeepImmune, DeepHLApan) with their associated metrics (R, Rank; BA, Binding Affinity; DAI, Differential Agretopicity Index; P, Percentile; S, Score) and thresholds (DOP, Distance to the Optimal Point; SB, Strong Binder; WB, Weak Binder; Author, Autor method suggested threshold) evaluated in the present study. (C) Evaluation pipeline with the used datasets and characteristics evaluated over each method. (D) Prediction performance for each method according to different performance metrics ranks. DOP, DOP rank; F1, F1-Score rank; FPR, False Positive Rate rank; FNR, False Negative Rate rank; FDRR, False Discovery Rates Rank; CR, Classification Rank; AR, Average Rank.

Figure 2

Benchmark results. (A) ROC curves for different software programs and direct metrics over mutated peptides. (B) Heatmap of methods predicted classification over neoantigens. BA, binding affinity; R, rank; S, score; DOP, distance to the optimal point; SB, strong binder. (C) Number of immunogenic peptides between: I top 10-ranked peptides, and II top 20-ranked peptides, according to each method. (D) I ROC curves for difference DAI and ratio DAI over anchor and non anchor position mutated peptides. II distribution of ratio DAI in logarithmic scale, for anchor and non-anchor position mutated peptides comparing negative and positive immunogenicity peptides. DAI, Differential Agretopicity Index. (E) P-values of association between TNB and clinical response to ICB distribution over ICB cohorts for TNB calculated according to each evaluated method. TNB, Tumor Neoantigen Burden; ICB, Immune Checkpoint Blockade.