Performance evaluation of commercial short-oligonucleotide microarrays and the impact of noise in making cross-platform correlations

Abstract

Background: Despite the widespread use of microarrays, much ambiguity regarding data analysis, interpretation and correlation of the different technologies exists. There is a considerable amount of interest in correlating results obtained between different microarray platforms. To date, only a few cross-platform evaluations have been published and unfortunately, no guidelines have been established on the best methods of making such correlations. To address this issue we conducted a thorough evaluation of two commercial microarray platforms to determine an appropriate methodology for making cross-platform correlations.

Results: In this study, expression measurements for 10,763 genes uniquely represented on Affymetrix U133A/B GeneChips and Amersham CodeLink UniSet Human 20 K microarrays were compared. For each microarray platform, five technical replicates, derived from the same total RNA samples, were labeled, hybridized, and quantified according to each manufacturers' standard protocols. The correlation coefficient (r) of differential expression ratios for the entire set of 10,763 overlapping genes was 0.62 between platforms. However, the correlation improved significantly (r = 0.79) when genes within noise were excluded. In addition to levels of inter-platform correlation, we evaluated precision, statistical-significance profiles, power, and noise levels for each microarray platform. Accuracy of differential expression was measured against real-time PCR for 25 genes and both platforms correlated well with r values of 0.92 and 0.79 for CodeLink and GeneChip, respectively.

Conclusions: As a result of this study, we recommend using only genes called 'present' in cross-platform correlations. However, as in this study, a large number of genes may be lost from the correlation due to differing levels of noise between platforms. This is an important consideration given the apparent difference in sensitivity of the two platforms. Data from microarray analysis need to be interpreted cautiously and therefore, we provide guidelines for making cross-platform correlations. In all, this study represents the most comprehensive and specifically designed comparison of short-oligonucleotide microarray platforms to date using the largest set of overlapping genes.

Figure 1

Pair-wise array precision of CodeLink and GeneChip with illustration of respective noise levels. The representative scatter plots show precision of normalized expression values relative to noise. All 10,763 overlapping gene probes are represented in these plots. Values highlighted in red were concordantly 'absent' (noise) calls on both arrays compared. Orange lines show two-fold limits, while the black line represents equality.

Figure 2

Coefficients of variation for each platform as a function of intensity, across all replicates. Genes which are concordantly 'absent' are shown in red. The black line represents the 100-probe moving average.

Figure 3

Correlation of differential expression ratios between CodeLink and GeneChip. Pearson correlation coefficients (r) are shown for each comparison. (A.) When all 10,763 overlapping genes are compared between platforms the correlation is 0.62. (B.) All values for genes concordantly 'absent' were removed prior to making the cross-platform correlation. In this case, 3,362 genes are called 'present' on at least 1 of the 5 replicates across both tissues and platforms. (C.) 2,569 genes called 'present' on at least 3 of the 5 replicates across both tissues and platforms. The correlation improves further to 0.74. (D.) Genes called present on all 5 replicates across both tissues and platforms. For these 1,760 genes the correlation is 0.79.

Figure 4

'Volcano plots' for CodeLink and GeneChip. Each point represents a gene from the uniquely common set of 10,763 genes between platforms. Data points highlighted in blue represent genes which are concordantly 'present' in both tissues. The log₁₀ ratio of expression (brain/pancreas) is shown on the x-axis and the p-value, from a two-tailed Student's t-tests on normalized log-transformed intensities, is shown on the y-axis. The vertical dashed lines represent 2-fold change ratios and the horizontal dashed line represents the statistical-significance level where p = 0.01.

Figure 5

Venn diagrams of differential expression calls and statistical significance across both microarray platforms. A two-sample two-tailed t-test on normalized log-transformed intensities was performed for each microarray platform. (A) The entire set of 10,763 uniquely common genes between platforms was used to determine the number of statistically significant (p < 0.01) expression ratios. Genes above and below noise were included in the analysis. (B) Statistical significance determined from the set of 2,569 genes which are 'present' on at least 3 arrays in both tissues. Expression values below noise ('absent') were not included in the analysis. (C) Statistical significance determined from the set of 1,760 genes which are 'present' on all 5 arrays in both tissues (i.e. concordantly 'present').

Figure 6

Power analysis estimating the number of technical array replicates needed to achieve a reasonable level of statistical power or confidence for CodeLink (blue) and GeneChip (red) when noise was included (solid diamonds) or excluded (open diamonds). For both graphs the alpha was set at 0.01. (A) Relationship between power and arrays necessary to statistically discriminate two-fold changes in expression. To achieve a power of 0.90 using all 10,763 genes, 3 arrays are minimally necessary for CodeLink while 8 are required for GeneChip. However, when noise is excluded, both GeneChip and CodeLink require only 1 array to achieve this same level of power. In fact, when noise is excluded, 1 array for both GeneChip and CodeLink has a power of 0.99 to detect two-fold changes in expression. B.) In order to detect 1.5 fold changes in expression, at a 0.90 power, when noise is excluded, CodeLink minimally requires 2 arrays while GeneChip requires 3.

Figure 7

Accuracy of CodeLink and GeneChip differential-expression ratios relative to qrtPCR. Expression ratios for each microarray platform were measured against results from qrtPCR for a randomly selected subset of 25 genes. Pearson correlation coefficients (r) are shown for each comparison.