Higher-order singular value decomposition: Difference between revisions

Revision as of 00:22, 16 June 2025

Revision as of 01:38, 16 June 2025

Line 1:

{{Short description|Tensor decomposition}}

In [[multilinear algebra]], the '''higher-order singular value decomposition''' ('''HOSVD''') of a [[tensor]] is a specific orthogonal [[Tucker decomposition]]. It generalizesmay be regarded as one type of generalization of the matrix [[singular value decomposition]]. to multi-way arrays andIt has applications in [[computer vision]], [[computer graphics]], [[machine learning]], [[scientific computing]], and [[signal processing]]. Some aspects can be traced as far back as [[F. L. Hitchcock]] in 1928,<ref name=":0">{{Cite journal|last=Hitchcock|first=Frank L|date=1928-04-01|title=Multiple Invariants and Generalized Rank of a M-Way Array or Tensor|journal=Journal of Mathematics and Physics|language=en|volume=7|issue=1–4|pages=39–79|doi=10.1002/sapm19287139|issn=1467-9590}}</ref> but it was [[L. R. Tucker]] who developed for third-order tensors the general [[Tucker decomposition]] in the 1960s,<ref name=":1">{{Cite journal|last=Tucker|first=Ledyard R.|date=1966-09-01|title=Some mathematical notes on three-mode factor analysis|journal=Psychometrika|language=en|volume=31|issue=3|pages=279–311|doi=10.1007/bf02289464|pmid=5221127|s2cid=44301099|issn=0033-3123}}</ref><ref name="Tucker1963">{{Cite journal|last=Tucker|first=L. R.|date=1963|title=Implications of factor analysis of three-way matrices for measurement of change|journal=In C. W. Harris (Ed.), Problems in Measuring Change. Madison, Wis.: Univ. Wis. Press.|pages=122–137}}</ref>

<ref name="Tucker1964">{{Cite journal|last=Tucker|first=L. R.|date=1964|title=The extension of factor analysis to three-dimensional matrices|journal=In N. Frederiksen and H. Gulliksen (Eds.), Contributions to Mathematical Psychology. New York: Holt, Rinehart and Winston|pages=109–127}}</ref> further advocated by [[Lieven De Lathauwer|L. De Lathauwer]] ''et al.''

Later, [[Lieven De Lathauwer]] ''et al.'' formalized the term HOSVD and proposed sequential algorithms based on the power method.<ref name=":2">{{Cite journal|last1=De Lathauwer|first1=L.|last2=De Moor|first2=B.|last3=Vandewalle|first3=J.|date=2000-01-01|title=On the Best Rank-1 and Rank-(R1 ,R2 ,. . .,RN) Approximation of Higher-Order Tensors|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1324–1342|doi=10.1137/S0895479898346995}}</ref><ref name="DeLathauwerSVD">{{Cite journal|last1issn=De Lathauwer0895-4798|first1citeseerx=L10.|last2=De Moor|first2=B1.|last3=Vandewalle|first3=J1.|date=2000-01-01|title=A Multilinear Singular Value Decomposition|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1253–1278|doi=10102.1137/s08954798963056969135}}</ref>

,<ref name="DeLathauwerSVD">{{Cite journal|last1=De Lathauwer|first1=L.|last2=De Moor|first2=B.|last3=Vandewalle|first3=J.|date=2000-01-01|title=A Multilinear Singular Value Decomposition|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1253–1278|doi=10.1137/s0895479896305696|issn=0895-4798|citeseerx=10.1.1.102.9135}}</ref> or advocated by Vasilescu and Terzopoulos.

The algorithm widely referred to in the literature as '''the Tucker''' or '''Higher-Order Singular Value Decomposition (HOSVD''' decomposition) was indeveloped factby Vasilescu and Terzopoulos, who introduced it under the name '''M-mode SVD''' by [[M.Alex O. Vasilescu]] and [[Demetri Terzopoulos]],<ref name=":Vasilescu2002">M. A. O. Vasilescu, D. Terzopoulos (2002), "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark. {{Webarchive|url=https://web.archive.org/web/20221229090931/http://www.cs.toronto.edu/~maov/tensorfaces/Springer%20ECCV%202002_files/eccv02proceeding_23500447.pdf |date=2022-12-29}}</ref><ref name="Vasilescu2003">M. A. O. Vasilescu, D. Terzopoulos (2003), "Multilinear Subspace Analysis of Image Ensembles," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’03), Madison, WI.</ref><ref name=":Vasilescu2005">M. A. O. Vasilescu, D. Terzopoulos (2005), "Multilinear Independent Component Analysis," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA.</ref> but hasfrequently sincemisattributed beento frequentlyeither misattributed[[L. R. TheTucker]] M-modeor SVD[[Lieven isDe aLathauwer]]. simpleWhile andthe elegantterm algorithmHOSVD suitablewas forcoined parallelby computation.De TheLathauwer, Tuckerit andis the HOSVDM-mode areSVD sequentialintroduced algorithmsby Vasilescu that employis gradientmost descentoften—and andincorrectly—referred theto powerunder method,that respectivelyname.

The conceptual foundations of tensor decompositions can be traced to [[F. L. Hitchcock]] (1928),<ref name=":0">{{Cite journal|last=Hitchcock|first=Frank L|date=1928-04-01|title=Multiple Invariants and Generalized Rank of a M-Way Array or Tensor|journal=Journal of Mathematics and Physics|language=en|volume=7|issue=1–4|pages=39–79|doi=10.1002/sapm19287139|issn=1467-9590}}</ref> and were significantly developed by [[L. R. Tucker]] in the 1960s, who introduced the general Tucker decomposition for third-order tensors.<ref name=":1">{{Cite journal|last=Tucker|first=Ledyard R.|date=1966-09-01|title=Some mathematical notes on three-mode factor analysis|journal=Psychometrika|volume=31|issue=3|pages=279–311|doi=10.1007/bf02289464}}</ref><ref name="Tucker1963">{{Cite journal|last=Tucker|first=L. R.|date=1963|title=Implications of factor analysis of three-way matrices for measurement of change|journal=In C. W. Harris (Ed.), Problems in Measuring Change. Madison, Wis.: Univ. Wis. Press.|pages=122–137}}</ref><ref name="Tucker1964">{{Cite journal|last=Tucker|first=L. R.|date=1964|title=The extension of factor analysis to three-dimensional matrices|journal=In N. Frederiksen and H. Gulliksen (Eds.), Contributions to Mathematical Psychology. New York: Holt, Rinehart and Winston|pages=109–127}}</ref>

The M-mode SVD is an algorithm suitable for parallel computation that applies the matrix SVD to compute orthonormal mode matrices, in contrast to the sequential algorithms proposed by Tucker that employs gradient descent, and De Lathauwer etal's algorithms that employs the power method. Vasilescu's parallel formulation is computationally distinct from earlier approaches.

Later, [[Lieven De Lathauwer]] ''et al.'' formalized the term HOSVD and proposed sequential algorithms based on the power method.<ref name=":2">{{Cite journal|last1=De Lathauwer|first1=L.|last2=De Moor|first2=B.|last3=Vandewalle|first3=J.|date=2000-01-01|title=On the Best Rank-1 and Rank-(R1 ,R2 ,. . .,RN) Approximation of Higher-Order Tensors|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1324–1342|doi=10.1137/S0895479898346995}}</ref><ref name="DeLathauwerSVD">{{Cite journal|last1=De Lathauwer|first1=L.|last2=De Moor|first2=B.|last3=Vandewalle|first3=J.|date=2000-01-01|title=A Multilinear Singular Value Decomposition|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1253–1278|doi=10.1137/s0895479896305696}}</ref>

The algorithm widely referred to as '''Tucker''' or '''HOSVD''' decomposition was in fact introduced under the name '''M-mode SVD''' by [[M.Alex O. Vasilescu]] and [[Demetri Terzopoulos]],<ref name=":Vasilescu2002">M. A. O. Vasilescu, D. Terzopoulos (2002), "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark. {{Webarchive|url=https://web.archive.org/web/20221229090931/http://www.cs.toronto.edu/~maov/tensorfaces/Springer%20ECCV%202002_files/eccv02proceeding_23500447.pdf |date=2022-12-29}}</ref><ref name="Vasilescu2003">M. A. O. Vasilescu, D. Terzopoulos (2003), "Multilinear Subspace Analysis of Image Ensembles," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’03), Madison, WI.</ref><ref name=":Vasilescu2005">M. A. O. Vasilescu, D. Terzopoulos (2005), "Multilinear Independent Component Analysis," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA.</ref> but has since been frequently misattributed. The M-mode SVD is a simple and elegant algorithm suitable for parallel computation. The Tucker and the HOSVD are sequential algorithms that employ gradient descent and the power method, respectively.

: This misattribution has had lasting impact on the scholarly record, obscuring the original source of a widely adopted algorithm, and complicating efforts to trace its development, reproduce results, or properly credit foundational contributions in multilinear algebra and tensor methods.