TCR-H: explainable machine learning prediction of T-cell receptor epitope binding on unseen datasets

Abstract

Artificial-intelligence and machine-learning (AI/ML) approaches to predicting T-cell receptor (TCR)-epitope specificity achieve high performance metrics on test datasets which include sequences that are also part of the training set but fail to generalize to test sets consisting of epitopes and TCRs that are absent from the training set, i.e., are 'unseen' during training of the ML model. We present TCR-H, a supervised classification Support Vector Machines model using physicochemical features trained on the largest dataset available to date using only experimentally validated non-binders as negative datapoints. TCR-H exhibits an area under the curve of the receiver-operator characteristic (AUC of ROC) of 0.87 for epitope 'hard splitting' (i.e., on test sets with all epitopes unseen during ML training), 0.92 for TCR hard splitting and 0.89 for 'strict splitting' in which neither the epitopes nor the TCRs in the test set are seen in the training data. Furthermore, we employ the SHAP (Shapley additive explanations) eXplainable AI (XAI) method for post hoc interrogation to interpret the models trained with different hard splits, shedding light on the key physiochemical features driving model predictions. TCR-H thus represents a significant step towards general applicability and explainability of epitope:TCR specificity prediction.

Keywords: T-cell receptor; adaptive immunity T-cell receptor; antigen; epitope; explainable machine learning; machine learning; physicochemical features; physicochemical model.

Figure 1

Schematic work flow of ML modeling. For each pair of TCR CDR3β and epitope sequences in the training data, labelled as binding and non-binding, physicochemical features are calculated and provided as input features for the different ML models tested. The trained models predict whether or not given CDR3β and epitope sequences of the test data bind. Models are evaluated based on a variety of performance metrics and are interpreted using SHAP analysis.

Figure 2

Performance metrics for epitope hard split test set of ML models. Random Forest (RF), Gradient Boosting trees (GBT), eXtreme Gradient Boosting (XGB), Support Vector Machines (SVM) and SVM with uncorrelated features (TCR-HE).

Figure 3

Performance metrics on the independent test sets for the TCR-HE (epitope hard split), TCR-Hβ (TCR hard split), TCR-HβE (strict split) and TCR-RS (random split).

Figure 4

AUC of ROC, Precision and Recall of TCR-HE compared with that of previously reported epitope hard split models (21, 23, 25, 36).

Figure 5

Representative summary plots of SHAP for Epitope split and TCR split showing top 50 features contributing to the model predictions. The length of the horizontal bar corresponding to each significant feature represents the magnitude of its SHAP value, while the color indicates the direction of its impact. Red bars denote higher predicted probabilities for the positive class, whereas blue bars represent the negative class. Longer bars against a feature indicate a greater impact on the model’s prediction.