Higher-order singular value decomposition: Difference between revisions

Content deleted Content added
No edit summary
Tag: Reverted
Line 1:
{{Short description|Tensor decomposition}}
In [[multilinear algebra]], the '''higher-order singular value decomposition''' ('''HOSVD''') of a [[tensor]] is a specific orthogonal [[Tucker decomposition]]. It may be regarded as one type of generalization ofgeneralizes the matrix [[singular value decomposition]]. Itto multi-way arrays and has applications in [[computer vision]], [[computer graphics]], [[machine learning]], [[scientific computing]], and [[signal processing]]. Some aspects can be traced as far back as [[F. L. Hitchcock]] in 1928,<ref name=":0">{{Cite journal|last=Hitchcock|first=Frank L|date=1928-04-01|title=Multiple Invariants and Generalized Rank of a M-Way Array or Tensor|journal=Journal of Mathematics and Physics|language=en|volume=7|issue=1–4|pages=39–79|doi=10.1002/sapm19287139|issn=1467-9590}}</ref> but it was [[L. R. Tucker]] who developed for third-order tensors the general [[Tucker decomposition]] in the 1960s,<ref name=":1">{{Cite journal|last=Tucker|first=Ledyard R.|date=1966-09-01|title=Some mathematical notes on three-mode factor analysis|journal=Psychometrika|language=en|volume=31|issue=3|pages=279–311|doi=10.1007/bf02289464|pmid=5221127|s2cid=44301099|issn=0033-3123}}</ref><ref name="Tucker1963">{{Cite journal|last=Tucker|first=L. R.|date=1963|title=Implications of factor analysis of three-way matrices for measurement of change|journal=In C. W. Harris (Ed.), Problems in Measuring Change. Madison, Wis.: Univ. Wis. Press.|pages=122–137}}</ref>
<ref name="Tucker1964">{{Cite journal|last=Tucker|first=L. R.|date=1964|title=The extension of factor analysis to three-dimensional matrices|journal=In N. Frederiksen and H. Gulliksen (Eds.), Contributions to Mathematical Psychology. New York: Holt, Rinehart and Winston|pages=109–127}}</ref> further advocated by [[Lieven De Lathauwer|L. De Lathauwer]] ''et al.''
<ref name=":2">{{Cite journal|last1=De Lathauwer|first1=L.|last2=De Moor|first2=B.|last3=Vandewalle|first3=J.|date=2000-01-01|title=On the Best Rank-1 and Rank-(R1 ,R2 ,. . .,RN) Approximation of Higher-Order Tensors|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1324–1342|doi=10.1137/S0895479898346995|issn=0895-4798|citeseerx=10.1.1.102.9135}}</ref>
,<ref name="DeLathauwerSVD">{{Cite journal|last1=De Lathauwer|first1=L.|last2=De Moor|first2=B.|last3=Vandewalle|first3=J.|date=2000-01-01|title=A Multilinear Singular Value Decomposition|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1253–1278|doi=10.1137/s0895479896305696|issn=0895-4798|citeseerx=10.1.1.102.9135}}</ref> or advocated by Vasilescu and Terzopoulos.
 
The conceptual foundations of tensor decompositions can be traced to [[F. L. Hitchcock]] (1928),<ref name=":0">{{Cite journal|last=Hitchcock|first=Frank L|date=1928-04-01|title=Multiple Invariants and Generalized Rank of a M-Way Array or Tensor|journal=Journal of Mathematics and Physics|language=en|volume=7|issue=1–4|pages=39–79|doi=10.1002/sapm19287139|issn=1467-9590}}</ref> and were significantly developed by [[L. R. Tucker]] in the 1960s, who introduced the general Tucker decomposition for third-order tensors.<ref name=":1">{{Cite journal|last=Tucker|first=Ledyard R.|date=1966-09-01|title=Some mathematical notes on three-mode factor analysis|journal=Psychometrika|volume=31|issue=3|pages=279–311|doi=10.1007/bf02289464}}</ref><ref name="Tucker1963">{{Cite journal|last=Tucker|first=L. R.|date=1963|title=Implications of factor analysis of three-way matrices for measurement of change|journal=In C. W. Harris (Ed.), Problems in Measuring Change. Madison, Wis.: Univ. Wis. Press.|pages=122–137}}</ref><ref name="Tucker1964">{{Cite journal|last=Tucker|first=L. R.|date=1964|title=The extension of factor analysis to three-dimensional matrices|journal=In N. Frederiksen and H. Gulliksen (Eds.), Contributions to Mathematical Psychology. New York: Holt, Rinehart and Winston|pages=109–127}}</ref>
The algorithm widely referred to in the literature as the Tucker or Higher-Order Singular Value Decomposition (HOSVD) was developed by Vasilescu and Terzopoulos, who introduced it under the name M-mode SVD,<ref name=":Vasilescu2002">M. A. O. Vasilescu, D. Terzopoulos (2002), "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark. {{Webarchive|url=https://web.archive.org/web/20221229090931/http://www.cs.toronto.edu/~maov/tensorfaces/Springer%20ECCV%202002_files/eccv02proceeding_23500447.pdf |date=2022-12-29}}</ref><ref name="Vasilescu2003">M. A. O. Vasilescu, D. Terzopoulos (2003), "Multilinear Subspace Analysis of Image Ensembles," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’03), Madison, WI.</ref><ref name=":Vasilescu2005">M. A. O. Vasilescu, D. Terzopoulos (2005), "Multilinear Independent Component Analysis," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA.</ref> but frequently misattributed to either [[L. R. Tucker]] or [[Lieven De Lathauwer]]. While the term HOSVD was coined by De Lathauwer, it is the M-mode SVD introduced by Vasilescu that is most often—and incorrectly—referred to under that name.
 
Later, [[Lieven De Lathauwer]] ''et al.'' formalized the term HOSVD and proposed sequential algorithms based on the power method.<ref name=":2">{{Cite journal|last1=De Lathauwer|first1=L.|last2=De Moor|first2=B.|last3=Vandewalle|first3=J.|date=2000-01-01|title=On the Best Rank-1 and Rank-(R1 ,R2 ,. . .,RN) Approximation of Higher-Order Tensors|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1324–1342|doi=10.1137/S0895479898346995}}</ref><ref name="DeLathauwerSVD">{{Cite journal|issnlast1=0895-4798De Lathauwer|citeseerxfirst1=10L.1|last2=De Moor|first2=B.1|last3=Vandewalle|first3=J.102|date=2000-01-01|title=A Multilinear Singular Value Decomposition|journal=SIAM Journal on Matrix Analysis and Applications|volume=21|issue=4|pages=1253–1278|doi=10.91351137/s0895479896305696}}</ref>
The M-mode SVD is an algorithm suitable for parallel computation that applies the matrix SVD to compute orthonormal mode matrices, in contrast to the sequential algorithms proposed by Tucker that employs gradient descent, and De Lathauwer etal's algorithms that employs the power method. Vasilescu's parallel formulation is computationally distinct from earlier approaches.
 
The algorithm widely referred to in the literature as the '''Tucker''' or Higher-Order Singular Value Decomposition ('''HOSVD)''' decomposition was developedin by Vasilescu and Terzopoulos, whofact introduced it under the name '''M-mode SVD''' by [[M.Alex O. Vasilescu]] and [[Demetri Terzopoulos]],<ref name=":Vasilescu2002">M. A. O. Vasilescu, D. Terzopoulos (2002), "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark. {{Webarchive|url=https://web.archive.org/web/20221229090931/http://www.cs.toronto.edu/~maov/tensorfaces/Springer%20ECCV%202002_files/eccv02proceeding_23500447.pdf |date=2022-12-29}}</ref><ref name="Vasilescu2003">M. A. O. Vasilescu, D. Terzopoulos (2003), "Multilinear Subspace Analysis of Image Ensembles," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’03), Madison, WI.</ref><ref name=":Vasilescu2005">M. A. O. Vasilescu, D. Terzopoulos (2005), "Multilinear Independent Component Analysis," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA.</ref> but frequentlyhas misattributedsince tobeen eitherfrequently [[Lmisattributed. R. Tucker]]The orM-mode [[LievenSVD Deis Lathauwer]].a Whilesimple theand termelegant HOSVDalgorithm wassuitable coinedfor byparallel Decomputation. Lathauwer,The itTucker isand the M-modeHOSVD SVDare introducedsequential by Vasilescualgorithms that isemploy mostgradient often—anddescent incorrectly—referredand tothe underpower thatmethod, namerespectively.
 
: This misattribution has had lasting impact on the scholarly record, obscuring the original source of a widely adopted algorithm, and complicating efforts to trace its development, reproduce results, or properly credit foundational contributions in multilinear algebra and tensor methods.