Cross-platform comparison of microarray-based multiple-class prediction

Abstract

High-throughput microarray technology has been widely applied in biological and medical decision-making research during the past decade. However, the diversity of platforms has made it a challenge to re-use and/or integrate datasets generated in different experiments or labs for constructing array-based diagnostic models. Using large toxicogenomics datasets generated using both Affymetrix and Agilent microarray platforms, we carried out a benchmark evaluation of cross-platform consistency in multiple-class prediction using three widely-used machine learning algorithms. After an initial assessment of model performance on different platforms, we evaluated whether predictive signature features selected in one platform could be directly used to train a model in the other platform and whether predictive models trained using data from one platform could predict datasets profiled using the other platform with comparable performance. Our results established that it is possible to successfully apply multiple-class prediction models across different commercial microarray platforms, offering a number of important benefits such as accelerating the possible translation of biomarkers identified with microarrays to clinically-validated assays. However, this investigation focuses on a technical platform comparison and is actually only the beginning of exploring cross-platform consistency. Further studies are needed to confirm the feasibility of microarray-based cross-platform prediction, especially using independent datasets.

Figure 1. Detailed information on the study design.

(a) Approach to development of the best classifier. (b) Assessment of performance of the best classifiers derived from different platforms. (c) Transferability of signature genes, i.e., whether predictive signature features selected in one platform could be directly used to train a model in the other platform. (d) Transferability of classifiers, i.e., whether predictive models trained using data from one platform could predict datasets profiled using the other platform with comparable performance.

Figure 2. Comparison of different platforms.

(a) Overall prediction accuracy for both test sets using models generated from each platform. Blue, yellow and brown bars represent ‘SeqMap’, ‘RefSeq’, and ‘Unigene’ for AFX, while corresponding circles faced green are for AGL. (b) Prediction accuracy for samples in each subclass using FKNN. (c) Prediction accuracy for samples in each subclass using LDA. (d) Prediction accuracy for samples in each subclass using SVM.

Figure 3. Transferability of predictive signature genes.

(a) Overall prediction accuracy for both test sets using signature genes selected from AFX (AFX to AGL). (b) Overall prediction accuracy for both test sets using signature genes selected from AGL (AGL to AFX). In (a) and (b), blue, yellow and brown bars represent ‘SeqMap’, ‘RefSeq’, and ‘Unigene’ for AFX, while corresponding circles faced green are for AGL. (c) Prediction accuracy for samples in each subclass using FKNN in the transfer of AFX to AGL. (d) Prediction accuracy for samples in each subclass using FKNN in the transfer of AGL to AFX.

Figure 4. Transferability of predictive classifiers.

(a) Overall prediction accuracy for both test sets using classifiers trained on AFX (AFX to AGL). (b) Overall prediction accuracy for both test sets using classifiers trained on AGL (AGL to AFX). In (a) and (b), blue, yellow and brown bars represent ‘SeqMap’, ‘RefSeq’, and ‘Unigene’ for AFX, while corresponding circles faced green are for AGL. (c) Prediction accuracy for samples in each subclass using FKNN in the transfer of AFX to AGL. (d) Prediction accuracy for samples in each subclass using FKNN in the transfer of AGL to AFX.