Dynamic kernel matching for non-conforming data: A case study of T cell receptor datasets

Abstract

Most statistical classifiers are designed to find patterns in data where numbers fit into rows and columns, like in a spreadsheet, but many kinds of data do not conform to this structure. To uncover patterns in non-conforming data, we describe an approach for modifying established statistical classifiers to handle non-conforming data, which we call dynamic kernel matching (DKM). As examples of non-conforming data, we consider (i) a dataset of T-cell receptor (TCR) sequences labelled by disease antigen and (ii) a dataset of sequenced TCR repertoires labelled by patient cytomegalovirus (CMV) serostatus, anticipating that both datasets contain signatures for diagnosing disease. We successfully fit statistical classifiers augmented with DKM to both datasets and report the performance on holdout data using standard metrics and metrics allowing for indeterminant diagnoses. Finally, we identify the patterns used by our statistical classifiers to generate predictions and show that these patterns agree with observations from experimental studies.

Fig 1. TCRs are examples of non-conforming data.

(a) Most statistical classifiers assume a fixed number of features (e.g. five) and that each feature represents the same kind of information across samples (e.g. shape). (b) These assumptions do not hold for non-conforming data. (c) A dataset of TCRs labelled by interaction with disease antigens. The dataset contains the amino acid symbols from regions of the TCR (CDR3) represented as sequences, which are examples of non-conforming data. (d) A dataset of TCR repertoires labelled by CMV serostatus. The dataset contains sequenced TCR repertoires represented as sets made of sequences, which is a different kind of non-conforming data than the previous dataset. (e) Samples are split into a training, validation, and test cohort (for panel c, identical sequences are first collapsed to ensure the same TCR does not appear in multiple cohorts). The training cohort is used to select the weights and biases of each model, the validation cohort is used for model selection, and the test cohort is used for reporting results.

Fig 2. Results on the antigen classification problem.

(a) The fit plotted for each weight update across the training (solid blue) and validation cohorts (solid red) steadily improves with each gradient optimization step. The fit to the test cohort after unblinding the samples (triangle) is significantly better than the baseline performance achievable by random chance (dashed black). (b) A confusion matrix of samples from the test cohort reveals the fraction of predictions that agree with the labels for each category (c) A 3D X-ray crystallographic structure of a TCR bound to GILGFVFTL:A0201. An alanine scan of the TCR CDR3 sequences reveals the largest |Δlogit| always corresponds to a contact position (asterisks).

Fig 3. Results on the repertoire Classification problem.

(a) The fit plotted for each weight update across the training (solid blue) and validation cohorts (solid red) steadily improves with each gradient optimization step. The fit to the test cohort after unblinding the samples (triangle) is better than the baseline performance achievable by random chance (dashed black). (b) A ROC curve reveals the sensitivity and specificity for various diagnostic thresholds of the model. (c) CDR3 sequences from receptors specific to various pMHCs are scored using the kernel of the model fitted in panel a. The receptors specific to a CMV peptide have the highest score, suggesting that the model has learned to identify receptors specific to this CMV peptide.

Fig 4. Results with confidence cutoffs.

The bar charts show the fraction of samples that are correctly classified (blue), incorrectly classified (red), and indeterminate (grey) (a) Using a 95% confidence cutoff determined on the validation cohort, the classification accuracy on the test cohort of the antigen classification problem is 97.06% capturing 44.5% of samples. The samples are not captured evenly across the six categories. (b) Using a 95% confidence cutoff determined using the validation cohort, the classification accuracy on the test cohort of the repertoire classification problem is 96.0% capturing 18.0% of samples. The samples are captured almost evenly across the two categories.