Assessing affymetrix GeneChip microarray quality

Abstract

Background: Microarray technology has become a widely used tool in the biological sciences. Over the past decade, the number of users has grown exponentially, and with the number of applications and secondary data analyses rapidly increasing, we expect this rate to continue. Various initiatives such as the External RNA Control Consortium (ERCC) and the MicroArray Quality Control (MAQC) project have explored ways to provide standards for the technology. For microarrays to become generally accepted as a reliable technology, statistical methods for assessing quality will be an indispensable component; however, there remains a lack of consensus in both defining and measuring microarray quality.

Results: We begin by providing a precise definition of microarray quality and reviewing existing Affymetrix GeneChip quality metrics in light of this definition. We show that the best-performing metrics require multiple arrays to be assessed simultaneously. While such multi-array quality metrics are adequate for bench science, as microarrays begin to be used in clinical settings, single-array quality metrics will be indispensable. To this end, we define a single-array version of one of the best multi-array quality metrics and show that this metric performs as well as the best multi-array metrics. We then use this new quality metric to assess the quality of microarry data available via the Gene Expression Omnibus (GEO) using more than 22,000 Affymetrix HGU133a and HGU133plus2 arrays from 809 studies.

Conclusions: We find that approximately 10 percent of these publicly available arrays are of poor quality. Moreover, the quality of microarray measurements varies greatly from hybridization to hybridization, study to study, and lab to lab, with some experiments producing unusable data. Many of the concepts described here are applicable to other high-throughput technologies.

Figure 1

Area under the ROC curve is increased by removing a poor quality array. ROC curves for detection of the 16 spiked-in transcripts using all 8 arrays (black line) and with each array removed (gray lines). The red ROC curve corresponds to removing array 4. Removing array 4 results in the largest increase in the area under the ROC curve suggesting that array 4 may be of poor quality.

Figure 2

Distances between replicate tissues. Histogram of the Euclidean distance between the gene expression values for paired tissue samples. We expect these distances to be fairly small because we assume that, for the most part, the same genes are expressed in samples from the same tissue. In fact, this does appear to be the case - the typical distance between replicate samples is 30. However, one pair of replicate tissues (Cardiac Myocytes) has a distance larger than 90, suggesting one of the samples may be of poor quality.

Figure 3

Removing poor quality arrays improves prediction. Plot of Matthews Correlation Coefficient for a clinical parameter, pathologic complete response, versus the number of lowest quality arrays removed for each quality metric. The prediction algorithm used was PAM. Prediction improved when removing the arrays with the poorest quality; however, some metrics did substantially better than others at detecting arrays that negatively affect prediction. RLE and Percent Present appeared to perform best, followed by NUSE and GNUSE. Average background showed no improvement when removing less than 30 arrays.

Figure 4

Distribution of GNUSE values. Histograms of GNUSE values from (A) 11299 HGU133a arrays from 338 studies and (B) 11029 HGU133plus2 arrays from 471 studies. Most GNUSE values are of acceptable quality (close to one), but the long right tail suggests some very poor quality probesets.

Figure 5

Distribution of median GNUSE values. Histograms of median GNUSE values from (A) 11299 HGU133a arrays or (B) 11029 HGU133plus2 arrays. The red vertical line represents the threshold of 1.25 - arrays with a median GNUSE greater than this threshold are considered poor quality. In both cases, this threshold appears to separate the majority of good quality arrays from the long right tail of poor quality arrays.

Figure 6

Lab accounts for more variability in quality than tissue type. Resulting from fitting Model 2. The individual random effects for lab and tissue are plotted and the estimated variance for each effect is reported. These estimates suggest that the lab in which an array was hybridized accounts for more of the variability in microarray data quality than the tissue that was hybridized to the array.