An experimentally derived confidence score for binary protein-protein interactions

Abstract

Information on protein-protein interactions is of central importance for many areas of biomedical research. At present no method exists to systematically and experimentally assess the quality of individual interactions reported in interaction mapping experiments. To provide a standardized confidence-scoring method that can be applied to tens of thousands of protein interactions, we have developed an interaction tool kit consisting of four complementary, high-throughput protein interaction assays. We benchmarked these assays against positive and random reference sets consisting of well documented pairs of interacting human proteins and randomly chosen protein pairs, respectively. A logistic regression model was trained using the data from these reference sets to combine the assay outputs and calculate the probability that any newly identified interaction pair is a true biophysical interaction once it has been tested in the tool kit. This general approach will allow a systematic and empirical assignment of confidence scores to all individual protein-protein interactions in interactome networks.

Figure 1

Strategy for deriving a confidence score for individual protein-protein interactions after HT screening using data from several complementary follow-up interaction assays. After initial screening using a HT platform, for example Y2H, all positives are evaluated using secondary tool-kit assays that have been benchmarked by PRS/RRS. The resulting raw data are integrated using a model trained on the PRS/RRS calibration data to derive a confidence based on the experimental support for each interaction (see text for details).

Figure 2

Schematic description of complementary tool-kit assays for binary protein interaction. (a) Yeast-2-hybrid: The Gal4 transcription factor is reconstituted to activate one or more independent reporter genes in S. cerevisiae. (b) MAPPIT: A bait-protein is fused to a hybrid erythropoietin-leptin receptor and the prey is fused to gp130. Upon stimulation with erythropoietin JAK2 molecules trans-phosphorylate each other and if bait and prey interact the activated JAKs will phosphorylate gp130, which in turn recruits and subsequently activates STAT3, which then activates transcription of a reporter (c) PCA: Two YFP fragments fused to bait and prey proteins reconstitute a fluorescent protein if brought in close proximity by two proteins that physically interact. (d) LUMIER: A luciferase tagged bait is co-expressed with a Flag-tagged prey in HEK-293T cells. The association between these proteins is determined by co-immunoprecipitation using an anti-Flag antibody and the presence of the bait is detected via its luciferase activity. (e) wNAPPA: Plasmids encoding GST-bait and HA-prey are mixed in a coupled transcription/translation reticulocyte lysate to express protein. Subsequently, the bait-GST is captured on the bottom of 96-well plate coated with anti-GST antibodies. If the proteins are interacting, the HA-prey can be immunologically detected.

Figure 3

Evaluation of assay performance at different stringencies using hsPRS-v1 and hsRRS-v1. (a) The tradeoff between true and false positive rate at different stringencies of the tool-kit assays. For different applications, different thresholds may be used. (b) Y2H assay performance in different scoring protocols (one vs. two reporters) and different bait and prey expression levels (high vs. low copy plasmid, y-axis). 3-AT (3-amino-1,2,4-triazole) is a competitive inhibitor of the HIS3 gene product and was included to reduce background in one set of experiments.

Figure 4

Performance of assays against positive and random reference sets (PRS and RRS). (a) Quantitation of assay sensitivity and specificity with standard error using hsPRS-v1 and hsRRS-v1. (b) Detection of individual hsPRS-v1 and hsRRS-v1 pairs by the tool-kit assays: Top panel: detected hsPRS-v1 pairs are indicated by green squares. Bottom panel detected hsRRS-v1 pairs are indicated by red squares. Phosphorylation dependent interactions are indicated by solid black frames. Thresholds used for the assays in this figure can be found in the Materials and Methods section.

Figure 5

Application of the integrated confidence score. (a) Application of the confidence scoring scheme to the Y2H-positive hsPRS-v1 to exemplify the process. Probabilities for pairs within each assay (middle panel) are computed using LOO-CV and a single-assay logistic regression model, trained identically to the combined-assay regression model. (b) Every reported interaction reported in future protein-protein interaction mapping experiments can be assigned a confidence score based on the tool-kit assay data for each individual protein pair.