Supervised multiway factorization

Abstract

We describe a probabilistic PARAFAC/CANDECOMP (CP) factorization for multiway (i.e., tensor) data that incorporates auxiliary covariates, SupCP. SupCP generalizes the supervised singular value decomposition (SupSVD) for vector-valued observations, to allow for observations that have the form of a matrix or higher-order array. Such data are increasingly encountered in biomedical research and other fields. We use a novel likelihood-based latent variable representation of the CP factorization, in which the latent variables are informed by additional covariates. We give conditions for identifiability, and develop an EM algorithm for simultaneous estimation of all model parameters. SupCP can be used for dimension reduction, capturing latent structures that are more accurate and interpretable due to covariate supervision. Moreover, SupCP specifies a full probability distribution for a multiway data observation with given covariate values, which can be used for predictive modeling. We conduct comprehensive simulations to evaluate the SupCP algorithm. We apply it to a facial image database with facial descriptors (e.g., smiling / not smiling) as covariates, and to a study of amino acid fluorescence. Software is available at https://github.com/lockEF/SupCP.

Keywords: Faces in the wild; dimension reduction; latent variables; parafac/candecomp; singular value decomposition; tensors.

Fig 1

Rank-1 components corresponding to Phenylalin, Tryptophan and Tryosine. The bottom two rows give the loadings for emission and excitation wavelengths, and the top row gives their resulting product.

Fig 2

Examples of frontalized faces.

Fig 3

Log-likelihood for the estimated SupCP and SupSVD models, for different ranks, on the training images used to fit the model and the test images.

Fig 4

Log-likelihood for the model over 2000 EM iterations. The first 500 iterations (a) incorporate random annealing to avoid local modes.

Fig 5

Constructed faces under different attribute covariates: SupCP.

Fig 6

Constructed faces under different attribute covariates: SupSVD.

Fig 7

True rank vs. estimated rank for 100 randomly generated simulations, under different levels of noise variance. A small amount of jitter is added to show multiple points with the same coordinates.