Multilinear principal component analysis: Difference between revisions

Revision as of 02:44, 26 May 2025

Revision as of 00:23, 17 June 2025

Line 1:

{{short description|Multilinear extension of principal component analysis}}

'''Multilinear principal component analysis''' ('''MPCA''') is a [[Multilinear algebra|multilinear]] extension of [[principal component analysis]] (PCA) that is used to analyze M-way arrays, also informally referred to as "data tensors". M-way arrays may be modeled by linear tensor models, such as CANDECOMP/Parafac, or by multilinear tensor models, such as multilinear principal component analysis (MPCA) or multilinear independent component analysis (MICA).

The origin of MPCA can be traced back to the [[tensorTensor rank decomposition]] were introduced by [[Frank Lauren Hitchcock]] in 1927;<ref>{{Cite journal

| author = F. L. Hitchcock

| author-link = F. L. Hitchcock

Line 11 ⟶ 12:

| issue = 1–4

| doi = 10.1002/sapm192761164

}}</ref> toexplanded upon with the [[Tucker decomposition]];<ref>{{Cite journal|last1=Tucker| first1=Ledyard R

| authorlink1 = Ledyard R Tucker

| title = Some mathematical notes on three-mode factor analysis

Line 18 ⟶ 19:

|date=September 1966

| doi = 10.1007/BF02289464 | pmid = 5221127

}}</ref> and toby Peter Kroonenberg'sthe "3-mode PCA" work.by Kroonenberg<ref name="Kroonenberg1980">P. M. Kroonenberg and J. de Leeuw, [https://doi.org/10.1007%2FBF02293599 Principal component analysis of three-mode data by means of alternating least squares algorithms], Psychometrika, 45 (1980), pp. 69–97.</ref> Kroonenbeg's algorithm is an itterative algorithm that employs gradient descent. In 2000, De Lathauwer et al. restated Tucker and Kroonenberg's work in clear and concise numerical computational terms in their SIAM paper entitled "[[Multilinear Singular Value Decomposition]]",<ref name="DeLathauwer2000a">{{cite journal | last1 = Lathauwer | first1 = L.D. | last2 = Moor | first2 = B.D. | last3 = Vandewalle | first3 = J. | year = 2000 | title = A multilinear singular value decomposition | url = http://portal.acm.org/citation.cfm?id=354398 | journal = SIAM Journal on Matrix Analysis and Applications | volume = 21 | issue = 4| pages = 1253–1278 | doi = 10.1137/s0895479896305696 | url-access = subscription }}</ref> (HOSVD) and provided an itterative algorithm that employed the power method in their paper "On the Best Rank-1 and Rank-(R₁, R₂, ..., R_N ) Approximation of Higher-order Tensors".<ref name=DeLathauwer2000b>{{cite journal | last1 = Lathauwer | first1 = L. D. | last2 = Moor | first2 = B. D. | last3 = Vandewalle | first3 = J. | year = 2000 | title = On the best rank-1 and rank-(R1, R2, ..., RN ) approximation of higher-order tensors | url = http://portal.acm.org/citation.cfm?id=354405 | journal = SIAM Journal on Matrix Analysis and Applications | volume = 21 | issue = 4| pages = 1324–1342 | doi = 10.1137/s0895479898346995 | url-access = subscription }}</ref>

CircaVasilescu and Terzopoulos in their paper "Multilinear Image Representation: TensorFaces"<ref name=Vasilescu2002a/> introduced the [[HOSVD| M-mode SVD]] algorithm which is a simple and elegant algorithm suitable for parallel computation. This algorithm is often misidentified in the literature as the HOSVD or the Tucker which are sequential itterative algorithms that employ gradient descent. 2001, Vasilescu and Terzopoulos reframedframed the data analysis, recognition and synthesis problems as multilinear tensor problems. TensorData factoris analysisviewed isas the compositional consequence of several causal factors of data formation, and which are well suited for multi-modal data tensor factor analysis. The power of the tensor framework was showcased by analyzing human motion joint angles, facial images or textures in terms of their causal factors of data formation in the following workspapers: Human Motion Signatures<ref name="Vasilescu2002b">M.A.O. Vasilescu (2002) [http://www.media.mit.edu/~maov/motionsignatures/hms_icpr02_corrected.pdf "Human Motion Signatures: Analysis, Synthesis, Recognition," Proceedings of International Conference on Pattern Recognition (ICPR 2002), Vol. 3, Quebec City, Canada, Aug, 2002, 456–460.]</ref>

(CVPR 2001, ICPR 2002), face recognition – [[TensorFaces]],<ref name="Vasilescu2002a"/><ref name="Vasilescu2003"/>

(ECCV 2002, CVPR 2003, etc.) and computer graphics – [[TensorTextures]]<ref name="Vasilescu2004"/> (Siggraph 2004).

Historically, MPCA has been referred to as "M-mode PCA", a terminology which was coined by Peter Kroonenberg in 1980.<ref name="Kroonenberg1980"/> In 2005, Vasilescu and [[Demetri Terzopoulos|Terzopoulos]] introduced the Multilinear PCA<ref name="MPCA-MICA2005">M. A. O. Vasilescu, D. Terzopoulos (2005) [http://www.media.mit.edu/~maov/mica/mica05.pdf "Multilinear Independent Component Analysis"], "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, June 2005, vol.1, 547–553."</ref> terminology as a way to better differentiate between linear and multilinear tensor decomposition, as well as, to better differentiate between the work<ref name="Vasilescu2002b"/><ref name="Vasilescu2002a"/><ref name="Vasilescu2003"/><ref name="Vasilescu2004"/> that computedemployed 2nd order statistics associated with each data tensor mode(axis), and subsequent work on Multilinear Independent Component Analysis<ref name="MPCA-MICA2005"/> that computedemployed higher order statistics associated with each tensor mode/axis.

Multilinear PCA may be applied to compute the causal factors of data formation, or as signal processing tool on data tensors whose individual observation have either been vectorized,<ref name="Vasilescu2002b"/><ref name="Vasilescu2002a">M.A.O. Vasilescu, [[Demetri Terzopoulos|D. Terzopoulos]] (2002) [http://www.media.mit.edu/~maov/tensorfaces/eccv02_corrected.pdf "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark, May, 2002, in Computer Vision – ECCV 2002, Lecture Notes in Computer Science, Vol. 2350, A. Heyden et al. (Eds.), Springer-Verlag, Berlin, 2002, 447–460. ]</ref><ref name="Vasilescu2003">M.A.O. Vasilescu, D. Terzopoulos (2003) [http://www.media.mit.edu/~maov/tensorfaces/cvpr03.pdf "Multilinear Subspace Analysis for Image Ensembles,'' M. A. O. Vasilescu, D. Terzopoulos, Proc. Computer Vision and Pattern Recognition Conf. (CVPR '03), Vol.2, Madison, WI, June, 2003, 93–99.]</ref><ref name="Vasilescu2004">M.A.O. Vasilescu, D. Terzopoulos (2004) [http://www.media.mit.edu/~maov/tensortextures/Vasilescu_siggraph04.pdf "TensorTextures: Multilinear Image-Based Rendering", M. A. O. Vasilescu and D. Terzopoulos, Proc. ACM SIGGRAPH 2004 Conference Los Angeles, CA, August, 2004, in Computer Graphics Proceedings, Annual Conference Series, 2004, 336–342. ]</ref> or whose observations are treated as a collection of column/row observations, an "dataobservation as a matrix", and concatenated into a data tensor. The main disadvantage of thisthe latter approach is that, rather than computing all possible combinations, MPCA computes a set of orthonormal matrices associated with each mode of the data tensor which are analogous to the orthonormal row and column space ofthat aare matrixunrelated computed byto the matrixcausal SVD. This transformation aims to capture as high a variance as possible, accounting for as muchfactors of the variability in the data associatedformation. with each data tensor mode(axis).

== The algorithm ==

The MPCA solution follows the alternating least square (ALS) approach. It is iterative in nature. As in PCA, MPCA works on centered data. Centering is a little more complicated for tensors, and it is problem dependent.

As in PCA, MPCA works on centered data. Centering is a little more complicated for tensors, and it is problem dependent.

== Feature selection ==