Higher-order singular value decomposition: Difference between revisions

Revision as of 13:06, 19 June 2025

Revision as of 13:35, 19 June 2025

Line 1:

{{Short description|Tensor decomposition}}

In [[multilinear algebra]], the '''higher-order singular value decomposition''' ('''HOSVD''') is a misnomer since the resulting tensor decomposition does not retain all the defining properties of the matrix SVD. The matrix SVD simultaneously yields a rank-𝑅 decomposition and orthonormal subspaces for both the row and column spaces. These properties are not realized within a single algorithm for higher-order tensors, but are instead distributedrealized acrossby two distinct algorithmic developments and two research directions. Harshman, as well as, the team of Carol and Chang proposed [[Canonical polyadic decomposition]] (CPD), which is a variant of the [[tensor rank decomposition]], in which the tensor is approximated as a sum of K rank-1 tensors for a user-specified ''K''. [[L. R. Tucker]] proposed a strategy for computing orthonormal subspaces for third order tensors. SomeAspecsts of these aspectsalgorithms can be traced as far back as [[F. L. Hitchcock]] in 1928.<ref name=":0">{{Cite journal|last=Hitchcock|first=Frank L|date=1928-04-01|title=Multiple Invariants and Generalized Rank of a M-Way Array or Tensor|journal=Journal of Mathematics and Physics|language=en|volume=7|issue=1–4|pages=39–79|doi=10.1002/sapm19287139|issn=1467-9590}}</ref>

[[Lieven De Lathauwer |De Lathauwer]] ''et al.''<ref name=":2">{{Cite journal|last1=De Lathauwer|first1=L.|last2=De Moor|first2=B.|last3=Vandewalle|first3=J.|date=2000-01-01|title=On the Best Rank-1 and Rank-(R1 ,R2 ,. . .,RN) Approximation of Higher-Order Tensors|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1324–1342|doi=10.1137/S0895479898346995|issn=0895-4798|citeseerx=10.1.1.102.9135}}</ref><ref name="DeLathauwerSVD">{{Cite journal|last1=De Lathauwer|first1=L.|last2=De Moor|first2=B.|last3=Vandewalle|first3=J.|date=2000-01-01|title=A Multilinear Singular Value Decomposition|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1253–1278|doi=10.1137/s0895479896305696|issn=0895-4798|citeseerx=10.1.1.102.9135}}</ref> introduced clarity to Tucker's concepts with two highly influential papers, while [[Vasilescu]] and [[Demetri Terzopoulos| Terzopoulos]] introduced algorithmic clarity.<ref name=":Vasilescu2002">M. A. O. Vasilescu, D. Terzopoulos (2002), "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark. {{Webarchive|url=https://web.archive.org/web/20221229090931/http://www.cs.toronto.edu/~maov/tensorfaces/Springer%20ECCV%202002_files/eccv02proceeding_23500447.pdf |date=2022-12-29}}</ref><ref name="Vasilescu2003">M. A. O. Vasilescu, D. Terzopoulos (2003), "Multilinear Subspace Analysis of Image Ensembles," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’03), Madison, WI.</ref><ref name=":Vasilescu2005">M. A. O. Vasilescu, D. Terzopoulos (2005), "Multilinear Independent Component Analysis," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA.</ref> They synthesized those ideas into an elegant two-step algorithm, one whose simplicity belies the complexity it resolves. Vasilescu and Terzopoulos introduced the '''M-mode SVD''' which is currently referred in the literature as the '''Tucker''' or the '''HOSVD'''. However, the Tucker algorithm, and De Lathauwer ''et al.'' algorithm are sequential, relying on iterative methods such as gradient descent or the power method, respectively. In contrast,The theterm '''M-mode SVD''' isaccurately reflects the algorithm; it captures the actual computation (a closedset of SVDs on mode-formflattenings) solutionwithout thatmaking assumptions about the structure of the core tensor. It can be computed sequentially, but is also well-suited for parallel computation.

: This misattribution has had lasting impact on the scholarly record, obscuring the original source of a widely adopted algorithm, and complicating efforts to trace its development, reproduce results, and recognizing the respective contributions of different research efforts.