Relative impact of key sources of systematic noise in Affymetrix and Illumina gene-expression microarray experiments

Abstract

Background: Systematic processing noise, which includes batch effects, is very common in microarray experiments but is often ignored despite its potential to confound or compromise experimental results. Compromised results are most likely when re-analysing or integrating datasets from public repositories due to the different conditions under which each dataset is generated. To better understand the relative noise-contributions of various factors in experimental-design, we assessed several Illumina and Affymetrix datasets for technical variation between replicate hybridisations of Universal Human Reference (UHRR) and individual or pooled breast-tumour RNA.

Results: A varying degree of systematic noise was observed in each of the datasets, however in all cases the relative amount of variation between standard control RNA replicates was found to be greatest at earlier points in the sample-preparation workflow. For example, 40.6% of the total variation in reported expressions were attributed to replicate extractions, compared to 13.9% due to amplification/labelling and 10.8% between replicate hybridisations. Deliberate probe-wise batch-correction methods were effective in reducing the magnitude of this variation, although the level of improvement was dependent on the sources of noise included in the model. Systematic noise introduced at the chip, run, and experiment levels of a combined Illumina dataset were found to be highly dependent upon the experimental design. Both UHRR and pools of RNA, which were derived from the samples of interest, modelled technical variation well although the pools were significantly better correlated (4% average improvement) and better emulated the effects of systematic noise, over all probes, than the UHRRs. The effect of this noise was not uniform over all probes, with low GC-content probes found to be more vulnerable to batch variation than probes with a higher GC-content.

Conclusions: The magnitude of systematic processing noise in a microarray experiment is variable across probes and experiments, however it is generally the case that procedures earlier in the sample-preparation workflow are liable to introduce the most noise. Careful experimental design is important to protect against noise, detailed meta-data should always be provided, and diagnostic procedures should be routinely performed prior to downstream analyses for the detection of bias in microarray studies.

Figure 1

Experiment designs. Schematic comparison of the sample pre-processing used in the generation of A: the previously published MCF7 and MCF10A Affymetrix GeneChips, B: our Illumina Ref-8/HT-12 data, and C: the MAQC Illumina/Affymetrix data.

Figure 2

Relative comparison of different batch effects in Affymetrix GeneChip data. LEFT, Comparison of the numbers of differentially expressed genes in common between MCF7 and MCF10A triplicate samples depending upon what variables were included in the model used for the ComBat correction ('amp' = amplification, 'lab' = labelling, and 'scn' = scanner). Red points are mean counts, error bars are standard deviations, and significance of increased counts compared to no ComBat correction are indicated ('**' for p < 0:05 and '***' for p < 0:01, based on two-tailed t-tests). RIGHT, Pairwise Pearson correlation heatmaps of MCF7 and MCF10A samples compensating for 1, 2, or 3 sources of batch effect using ComBat. Green and blue colour-bars denote MCF7 and MCF10A samples, respectively. The lightest colours denote un-amplified samples, slightly darker are amplified samples, and the darkest colours are the scanner/labelling comparison. A, all data treated as a single group without batch-correction; B, batch-correction for amplification; C, batch-correction for amplification and an alternative labelling method; D, batch correction for amplification, labelling and different scanners used.

Figure 3

Comparison of variance components in Affymetrix and Illumina data. Comparison of MAQC and Paterson Affymetrix variance components. A: Probe-wise SD-estimates corresponding to several levels of technical variance (see figure key) in detection-filtered and quantile normalised MCF7/MCF10 expressions from the Paterson Affymetrix dataset. B: Probe-wise estimates of standard deviations (SD) corresponding to the inter-experiment (light blue), inter-run (dark blue), and inter-chip (green) technical variance in our UHRR Illumina data. The effect on these standard deviations following detection-filtering (DF), quantile-normalisation (QN), ComBat batch-correction by experiment (CB(expt)), and ComBat batch-correction by run (CB(run)) are shown. C: Probe-wise SD-estimates corresponding to inter-laboratory (pale blue), inter-chip (green), and inter-array (dark green) technical variances in detection-filtered and quantile normalised UHRR/UBRR expressions from the MAQC Illumina dataset. D: Probe-wise SD-estimates corresponding to inter-laboratory (pale blue) and inter-chip (green) technical variances in detection-filtered and quantile normalised UHRR/UBRR expressions from the MAQC Affymetrix dataset.

Figure 4

Correlation heatmap of all replicate UHRR pairs. Heatmap of Pearson correlations between replicate pairs of UHRR samples highlights the inter-experiment, inter-run, and inter-chip differences; particularly at the inter-experiment level. Red cells correspond to ~80% correlation and white to 100% correlation. Batches and sample numbers are consistent with the colouring and labelling in Figures 1 and Additional File 2. A: detection filtered (DF); B: DF & quantile normalised (QN); C: DF & QN &ComBat(by experiment); D: DF & QN &ComBat(by run).

Figure 5

Correlation heatmap of all replicate pool pairs. RIGHT: Heatmap of Pearson correlations between replicate pairs of pooled-tumour control samples highlights the inter-run and inter-chip variation; particularly at the inter-chip level. Red cells correspond to ~96% correlation and white to 100% correlation. Batches and sample numbers are consistent with the colouring and labelling in Figures 1 and Additional File 2. A: detection filtered (DF); B: DF & quantile normalised (QN); C: DF & QN &ComBat(by run); D: DF & QN &ComBat(by chip). LEFT: Variance estimates at various levels in replicate pooled-controls hybridised to the HT-12 chips used in experiment 2. Highlights the effect of various normalisation procedures on these variance estimates; as before, such procedures include detection-filtered (DF), quantile normalised (QN), ComBat corrected by run (CB(run)), and ComBat corrected by BeadChip (CB(chip)).

Figure 6

Batch effect and probe GC content. TOP: A, histogram of GC content over our Illumina HT-12 probes; B, gaussian-smoothed probability density distribution (black, N = 48,803). Union of probes more than 2-fold up- or down-regulated due to the batch effect in any of the 14 duplicate tumour-sample pairs from Additional File 4 (green line, N = 2,661). Union of probes more than 2-fold up- or down-regulated due to the batch effect in more than 5 of the duplicate tumour pairs (red line, N = 207). BOTTOM: Plots of probe CG-fraction against standard deviation estimated at the inter-experiment (A), inter-run (B), and inter-chip (C) levels in our combined Illumina Ref-8/HT-12 dataset. Red lines denote the cutoff s used in chi-squared analysis at each level.