Multilinear principal component analysis: Difference between revisions

Revision as of 05:13, 20 June 2025

Latest revision as of 05:40, 20 June 2025

Line 1:

{{short description|Multilinear extension of principal component analysis}}

'''Multilinear principal component analysis''' ('''MPCA''') is a [[Multilinear algebra|multilinear]] extension of [[principal component analysis]] (PCA) that is used to analyze M-way arrays, also informally referred to as "data tensors". M-way arrays may be modeled by
* '''linear tensor models''', such as [[Tensor rank|CANDECOMP/Parafac]], or by
* '''multilinear tensor models''', such as '''multilinear principal component analysis (MPCA)'''<ref name=Vasilescu2002a/><ref name=Vasilescu2003/> or '''multilinear (tensor) independent component analysis (MICA)'''.<ref name=MPCA-MICA2005/>

In 2005, [[Vasilescu]] and [[Demetri Terzopoulos|Terzopoulos]] introduced the '''Multilinear PCA'''<ref name="MPCA-MICA2005">M. A. O. Vasilescu, D. Terzopoulos (2005) [http://www.media.mit.edu/~maov/mica/mica05.pdf "Multilinear Independent Component Analysis"], "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, June 2005, vol.1, 547–553."</ref> terminology as a way to better differentiate between linear and multilinear tensor decomposition, as well as, to better differentiate between data models that employed 2nd order statistics associated with each data tensor mode(axis),<ref name="Vasilescu2002b"/><ref name="Vasilescu2002a"/><ref name="Vasilescu2003"/><ref name="Vasilescu2004"/> and subsequent work on '''Multilinear Independent Component Analysis'''<ref name="MPCA-MICA2005"/> that employedversus higher order statistics associated with each tensor mode/axis andto computedcompute a set of independent components for each mode., such as '''Multilinear ICA'''<ref name="MPCA-MICA2005"/>

Multilinear PCA may be applied to compute the causal factors of data formation, or as signal processing tool on data tensors whose individual observation have either been vectorized,<ref name="Vasilescu2002b"/><ref name="Vasilescu2002a">M.A.O. Vasilescu, [[Demetri Terzopoulos|D. Terzopoulos]] (2002) [http://www.media.mit.edu/~maov/tensorfaces/eccv02_corrected.pdf "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark, May, 2002, in Computer Vision – ECCV 2002, Lecture Notes in Computer Science, Vol. 2350, A. Heyden et al. (Eds.), Springer-Verlag, Berlin, 2002, 447–460. ]</ref><ref name="Vasilescu2003">M.A.O. Vasilescu, D. Terzopoulos (2003) [http://www.media.mit.edu/~maov/tensorfaces/cvpr03.pdf "Multilinear Subspace Analysis for Image Ensembles,'' M. A. O. Vasilescu, D. Terzopoulos, Proc. Computer Vision and Pattern Recognition Conf. (CVPR '03), Vol.2, Madison, WI, June, 2003, 93–99.]</ref><ref name="Vasilescu2004">M.A.O. Vasilescu, D. Terzopoulos (2004) [http://www.media.mit.edu/~maov/tensortextures/Vasilescu_siggraph04.pdf "TensorTextures: Multilinear Image-Based Rendering", M. A. O. Vasilescu and D. Terzopoulos, Proc. ACM SIGGRAPH 2004 Conference Los Angeles, CA, August, 2004, in Computer Graphics Proceedings, Annual Conference Series, 2004, 336–342. ]</ref> or whose observations are treated as a collection of column/row observations, an "observation as a matrix", and concatenated into a data tensor. The main disadvantage of the latter approach is thatsuitable MPCAfor computescompression aand setreducing ofredundancy orthonormalin matricesthe associatedrows, with rowcolumns and column spacefibers that are unrelated to the causal factors of data formation.

Vasilescu and Terzopoulos in their paper "[[Multilinear Image Representation: TensorFaces]]"<ref name=Vasilescu2002a/><ref name="Vasilescu2003"/> introduced the [[HOSVD| '''M-mode SVD''']] algorithm which is a simple and elegant algorithm that is oftenare algorithms misidentified in the literature as the '''HOSVD'''<ref name=DeLathauwer2000b>{{cite journal | last1 = Lathauwer | first1 = L. D. | last2 = Moor | first2 = B. D. | last3 = Vandewalle | first3 = J. | year = 2000 | title = On the best rank-1 and rank-(R1, R2, ..., RN ) approximation of higher-order tensors | url = http://portal.acm.org/citation.cfm?id=354405 | journal = SIAM Journal on Matrix Analysis and Applications | volume = 21 | issue = 4| pages = 1324–1342 | doi = 10.1137/s0895479898346995 | url-access = subscription }}</ref><ref name="DeLathauwer2000a">{{cite journal | last1 = Lathauwer | first1 = L.D. | last2 = Moor | first2 = B.D. | last3 = Vandewalle | first3 = J. | year = 2000 | title = A multilinear singular value decomposition | url = http://portal.acm.org/citation.cfm?id=354398 | journal = SIAM Journal on Matrix Analysis and Applications | volume = 21 | issue = 4| pages = 1253–1278 | doi = 10.1137/s0895479896305696 | url-access = subscription }}</ref>

or the '''Tucker''' which employ the power method or gradient descent, respectively.

or the '''Tucker''' which are sequential itterative algorithms that employ the power method or gradient descent, respectively. Vasilescu and Terzopoulos framed the data analysis, recognition and synthesis problems as multilinear tensor problems. Data is viewed as the compositional consequence of several causal factors, and which are well suited for multi-modal tensor factor analysis. The power of the tensor framework was showcased by analyzing human motion joint angles, facial images or textures in the following papers: Human Motion Signatures<ref name="Vasilescu2002b">M.A.O. Vasilescu (2002) [http://www.media.mit.edu/~maov/motionsignatures/hms_icpr02_corrected.pdf "Human Motion Signatures: Analysis, Synthesis, Recognition," Proceedings of International Conference on Pattern Recognition (ICPR 2002), Vol. 3, Quebec City, Canada, Aug, 2002, 456–460.]</ref>

or the '''Tucker''' which are sequential itterative algorithms that employ the power method or gradient descent, respectively. Vasilescu and Terzopoulos framed the data analysis, recognition and synthesis problems as multilinear tensor problems. Data is viewed as the compositional consequence of several causal factors, and whichthat are well suited for multi-modal tensor factor analysis. The power of the tensor framework was showcased by analyzing human motion joint angles, facial images or textures in the following papers: Human Motion Signatures<ref name="Vasilescu2002b">M.A.O. Vasilescu (2002) [http://www.media.mit.edu/~maov/motionsignatures/hms_icpr02_corrected.pdf "Human Motion Signatures: Analysis, Synthesis, Recognition," Proceedings of International Conference on Pattern Recognition (ICPR 2002), Vol. 3, Quebec City, Canada, Aug, 2002, 456–460.]</ref>

(CVPR 2001, ICPR 2002), face recognition – [[TensorFaces]],<ref name="Vasilescu2002a"/><ref name="Vasilescu2003"/>

(ECCV 2002, CVPR 2003, etc.) and computer graphics – [[TensorTextures]]<ref name="Vasilescu2004"/> (Siggraph 2004).