A measure of the signal-to-noise ratio of microarray samples and studies using gene correlations

Abstract

Background: The quality of gene expression data can vary dramatically from platform to platform, study to study, and sample to sample. As reliable statistical analysis rests on reliable data, determining such quality is of the utmost importance. Quality measures to spot problematic samples exist, but they are platform-specific, and cannot be used to compare studies.

Results: As a proxy for quality, we propose a signal-to-noise ratio for microarray data, the "Signal-to-Noise Applied to Gene Expression Experiments", or SNAGEE. SNAGEE is based on the consistency of gene-gene correlations. We applied SNAGEE to a compendium of 80 large datasets on 37 platforms, for a total of 24,380 samples, and assessed the signal-to-noise ratio of studies and samples. This allowed us to discover serious issues with three studies. We show that signal-to-noise ratios of both studies and samples are linked to the statistical significance of the biological results.

Conclusions: We showed that SNAGEE is an effective way to measure data quality for most types of gene expression studies, and that it often outperforms existing techniques. Furthermore, SNAGEE is platform-independent and does not require raw data files. The SNAGEE R package is available in BioConductor.

Figure 1. Link between statistical significance and study SNR in NCI60 datasets.

Statistical significance is estimated as the fraction of differentially expressed genes (%DEG) between a cancer type and the other cancer types, each panel representing a different cancer type. Higher SNR leads to higher %DEG, as expected. are on Affymetrix platforms (GSE5720 and GSE5949), is on NCI dual channel (GSE2003) and is on Stanford dual channel (GSE7947). c.c. are correlation coefficients.

Figure 2. Effect of increasing levels of noise on study SNR and statistical significance.

Statistical significance was estimated as the fraction of differentially expressed genes (%DEG). Noise was added to a study of the NCI60 on the U133A platform (GSE5720). SNR and %DEG were calculated on the modified study. Increasing noise lead to lower SNR and lower %DEG. c.c. are correlation coefficients, which underestimate the strength of the relations as those are not linear.

Figure 3. Effect of biological outliers on study SNR.

The study SNRs were calculated in function of the number of biological outliers added to a study consiting of homogeneous samples. (A) Outliers are cell lines, original study consists of tumor and normal tissues. (B) Outliers are ESR− breast cancers, original study consists of ESR+ breast cancers.

Figure 4. SNR in function of the number of samples: effect of disattenuation.

Subsets of a very large dataset (expO) were created by randomly selecting a number of samples. The study SNRs of those subsets are shown in function of their size, with disattenuation () or without (). The error bars are the 95% confidence intervals determined by resampling. The horizontal line is the SNR of the whole data set.

Figure 5. Comparison of quality metrics with Bayes classifier posterior probabilities.

Samples were classified by a naive Bayes classifier. Bad quality samples should not fit the classification well, and have lower probabilities. The average of the ranks of the posterior probabilities of the lowest quality samples is shown. Quality of the samples was determined with SNAGEE (solid line), MDQC (dashed line), GNUSE (dotted line), NUSE (grey dotted line) and RLE (dash-dot line). Classification was done on breast cancers according to ER status (A:GSE20194 and B:GSE4922); on muscle disease samples according to gender (C) or to the muscle disease type (D), one class vs. the rest. The solid grey line uses the samples assigned the highest SNR (and so likely to be biological outliers) by SNAGEE.

Figure 6. SNR of samples as biological outliers.

The SNR of the samples within the complete study (x-axis) are compared with their SNR when removing all similar samples (y-axis). A–M. Muscle diseases, N. Cell lines vs. normal and diseased tissues.