Higher-order singular value decomposition: Difference between revisions

Revision as of 03:25, 19 June 2025

Revision as of 03:26, 19 June 2025

Line 1:

{{Short description|Tensor decomposition}}

In [[multilinear algebra]], the '''higher-order singular value decomposition''' ('''HOSVD''') is a misnomer since the resulting tensor decomposition does not retain all the defining properties of the matrix SVD. The matrix SVD simultaneously yields a rank-𝑅 decomposition and orthonormal subspaces for both the row and column spaces. These properties are not realized within a single algorithm for higher-order tensors, but are instead distributed across two distinct algorithmic developments and two research directions. Harshman, as well as, the team of Carol and Chang proposed Canonical polyadic decomposition (CPD), which is a variant of the [[tensor rank decomposition]], in which the tensor is approximated as a sum of K rank-1 tensors for a user-specified K. [[L. R. Tucker]] proposed a strategy for computing orthonormal subspaces for third order tensors. Some aspects can be traced as far back as [[F. L. Hitchcock]] in 1928,.<ref name=":0">{{Cite journal|last=Hitchcock|first=Frank L|date=1928-04-01|title=Multiple Invariants and Generalized Rank of a M-Way Array or Tensor|journal=Journal of Mathematics and Physics|language=en|volume=7|issue=1–4|pages=39–79|doi=10.1002/sapm19287139|issn=1467-9590}}</ref>

[[Lieven De Lathauwer |De Lathauwer]] ''et al.''<ref name=":2">{{Cite journal|last1=De Lathauwer|first1=L.|last2=De Moor|first2=B.|last3=Vandewalle|first3=J.|date=2000-01-01|title=On the Best Rank-1 and Rank-(R1 ,R2 ,. . .,RN) Approximation of Higher-Order Tensors|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1324–1342|doi=10.1137/S0895479898346995|issn=0895-4798|citeseerx=10.1.1.102.9135}}</ref><ref name="DeLathauwerSVD">{{Cite journal|last1=De Lathauwer|first1=L.|last2=De Moor|first2=B.|last3=Vandewalle|first3=J.|date=2000-01-01|title=A Multilinear Singular Value Decomposition|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1253–1278|doi=10.1137/s0895479896305696|issn=0895-4798|citeseerx=10.1.1.102.9135}}</ref> provided clarity to Tucker's concepts with two seminalhighly influential papers, while [[Vasilescu]] and [[Demetri Terzopoulos| Terzopoulos]] provided algorithmic clarity. They synthesized those ideas into an elegant two-step algorithm, one whose simplicity belies the complexity it resolves.<ref name=":Vasilescu2002">M. A. O. Vasilescu, D. Terzopoulos (2002), "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark. {{Webarchive|url=https://web.archive.org/web/20221229090931/http://www.cs.toronto.edu/~maov/tensorfaces/Springer%20ECCV%202002_files/eccv02proceeding_23500447.pdf |date=2022-12-29}}</ref><ref name="Vasilescu2003">M. A. O. Vasilescu, D. Terzopoulos (2003), "Multilinear Subspace Analysis of Image Ensembles," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’03), Madison, WI.</ref><ref name=":Vasilescu2005">M. A. O. Vasilescu, D. Terzopoulos (2005), "Multilinear Independent Component Analysis," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA.</ref> Vasilescu and Terzopoulos introduced the '''M-mode SVD''' which is currently referred in the literature as the '''Tucker''' or the '''HOSVD'''. However, the Tucker/Kroonenberg algorithm, and De Lathauwer ''et al.'' algorithms are sequential, relying on iterative methods such as gradient descent or the power method, respectively. In contrast, the M-mode SVD is a closed-form solution that can be computed sequentially, but is also well-suited for parallel computation.

: This misattribution has had lasting impact on the scholarly record, obscuring the original source of a widely adopted algorithm, and complicating efforts to trace its development, reproduce results, and recognizing the respective contributions of different research efforts.