PCA as a practical indicator of OPLS-DA model reliability

Abstract

Background: Principal Component Analysis (PCA) and Orthogonal Projections to Latent Structures Discriminant Analysis (OPLS-DA) are powerful statistical modeling tools that provide insights into separations between experimental groups based on high-dimensional spectral measurements from NMR, MS or other analytical instrumentation. However, when used without validation, these tools may lead investigators to statistically unreliable conclusions. This danger is especially real for Partial Least Squares (PLS) and OPLS, which aggressively force separations between experimental groups. As a result, OPLS-DA is often used as an alternative method when PCA fails to expose group separation, but this practice is highly dangerous. Without rigorous validation, OPLS-DA can easily yield statistically unreliable group separation.

Methods: A Monte Carlo analysis of PCA group separations and OPLS-DA cross-validation metrics was performed on NMR datasets with statistically significant separations in scores-space. A linearly increasing amount of Gaussian noise was added to each data matrix followed by the construction and validation of PCA and OPLS-DA models.

Results: With increasing added noise, the PCA scores-space distance between groups rapidly decreased and the OPLS-DA cross-validation statistics simultaneously deteriorated. A decrease in correlation between the estimated loadings (added noise) and the true (original) loadings was also observed. While the validity of the OPLS-DA model diminished with increasing added noise, the group separation in scores-space remained basically unaffected.

Conclusion: Supported by the results of Monte Carlo analyses of PCA group separations and OPLS-DA cross-validation metrics, we provide practical guidelines and cross-validatory recommendations for reliable inference from PCA and OPLS-DA models.

Keywords: Chemometrics; Metabolomics; OPLS; PCA; PLS.

Fig. 1

Relationships to OPLS-DA CV-ANOVA p values obtained through Monte Carlo simulation of the Mahalanobis distance (D_M) between classes in PCA scores-space. Panels (A) and (B) hold results computed from the Coffees and Media datasets, respectively. The density of points in both panels is indicated by coloring, where red indicates high point density and blue indicates low density.

Fig. 2

Relationships to OPLS-DA CV-ANOVA p values obtained through Monte Carlo simulation of correlation between OPLS-DA model predictive loadings given noisy data (p) and loadings obtained on the original data matrix (p₀). Panels (A) and (B) hold results computed from the Coffees and Media datasets, respectively. The density of points in both panels is indicated by coloring, where red indicates high point density and blue indicates low density.

Fig. 3

(A) Decrease of correlation between estimated loadings (p) and true loadings (p₀) occurs as varying degrees of noise are added to the Coffees (red) and Media (blue) data matrices. Light shaded regions indicate confidence intervals of plus or minus one standard deviation from the mean correlation. A value of 1X additive noise corresponds to a noise standard deviation equaling 0.002 times the data matrix l₂ norm. (B) Increase of p values from CV-ANOVA OPLS-DA validation as varying degrees of noise are added to the Coffees (red) and Media (blue) data matrices. Light shaded regions indicate confidence intervals of plus or minus one standard deviation from the median p value.

Fig. 4

Comparison of representative PCA (A, C, E) and OPLS-DA (B, D, F) scores resulting from modeling the original Coffees data matrix (A, B), the 4X noisy data matrix (C, D) and the 20X noisy data matrix (E, F). Class ellipses represent the 95% confidence regions for class membership. CV-ANOVA p-values for the OPLS-DA model generated from the original data matrix, 4X and 20X noisy data matrix are 2.82x10⁻¹¹, 2.99x10⁻⁴, and 1.97x10⁻¹, respectively.