Determining confidence of predicted interactions between HIV-1 and human proteins using conformal method

Abstract

Identifying protein-protein interactions (PPI's) is critical for understanding virtually all cellular molecular mechanisms. Previously, predicting PPI's was treated as a binary classification task and has commonly been solved in a supervised setting which requires a positive labeled set of known PPI's and a negative labeled set of non-interacting protein pairs. In those methods, the learner provides the likelihood of the predicted interaction, but without a confidence level associated with each prediction. Here, we apply a conformal prediction framework to make predictions and estimate confidence of the predictions. The conformal predictor uses a function measuring relative 'strangeness' interacting pairs to check whether prediction of a new example added to the sequence of already known PPI's would conform to the 'exchangeability' assumption: distribution of interacting pairs is invariant with any permutations of the pairs. In fact, this is the only assumption we make about the data. Another advantage is that the user can control a number of errors by providing a desirable confidence level. This feature of CP is very useful for a ranking list of possible interactive pairs. In this paper, the conformal method has been developed to deal with just one class - class interactive proteins - while there is not clearly defined of 'non-interactive'pairs. The confidence level helps the biologist in the interpretation of the results, and better assists the choices of pairs for experimental validation. We apply the proposed conformal framework to improve the identification of interacting pairs between HIV-1 and human proteins.

Fig. 1

Random Forest (RF) prediction scores (where they are positive) and p-values obtained by the conformal predictor for the same pairs. Crosses (interactions) are known interactions, other examples are represented as points. Examples with RF prediction score less than zero are not included. The horizontal line corresponds to 10% cut-off for p-value (significance level), or confidence level 90% (for a true interaction, we are 90% sure that it is above this line).

Fig. 2

Dependence of the size of conformal prediction list on the confidence level. Lower line corresponds to the percentage of pairs predicted as interactions both by conformal prediction and by positive random forest score.