Matrix decomposition: Difference between revisions

Revision as of 20:09, 19 May 2022

Latest revision as of 06:53, 18 July 2025

(23 intermediate revisions by 15 users not shown)

Line 6:

In [[numerical analysis]], different decompositions are used to implement efficient matrix [[algorithm]]s.

For instanceexample, when solving a [[system of linear equations]] <math>A \mathbf{x} = \mathbf{b}</math>, the matrix ''A'' can be decomposed via the [[LU decomposition]]. The LU decomposition factorizes a matrix into a [[lower triangular matrix]] ''L'' and an [[upper triangular matrix]] ''U''. The systems <math>L(U \mathbf{x}) = \mathbf{b}</math> and <math>U \mathbf{x} = L^{-1} \mathbf{b}</math> require fewer additions and multiplications to solve, compared with the original system <math>A \mathbf{x} = \mathbf{b}</math>, though one might require significantly more digits in inexact arithmetic such as [[floating point]].

Similarly, the [[QR decomposition]] expresses ''A'' as ''QR'' with ''Q'' an [[orthogonal matrix]] and ''R'' an upper triangular matrix. The system ''Q''(''R'''''x''') = '''b''' is solved by ''R'''''x''' = ''Q''^T'''b''' = '''c''', and the system ''R'''''x''' = '''c''' is solved by '[[Triangular matrix#Forward and back substitution|back substitution]]'. The number of additions and multiplications required is about twice that of using the LU solver, but no more digits are required in inexact arithmetic because the QR decomposition is [[numerically stable]].

Line 14:

=== LU decomposition ===

{{main|LU decomposition}}

*Traditionally applicable to: [[square matrix]] ''A'', although rectangular matrices can be applicable.<ref>{{Cite book|last=Lay|first=David C.|url=https://www.worldcat.org/oclc/920463015|title=Linear algebra and its applications|date=2016|others=Steven R. Lay, Judith McDonald|isbn=978-1-292-09223-82|edition=Fifth Global|___location=Harlow|pages=142|oclc=920463015}}</ref><ref group="nb">If a non-square matrix is used, however, then the matrix ''U'' will also have the same rectangular shape as the original matrix ''A''. And so, calling the matrix ''U'' upper triangular would be incorrect as the correct term would be that ''U'' is the 'row echelon form' of ''A''. Other than this, there are no differences in LU factorization for square and non-square matrices.</ref>

*Decomposition: <math>A=LU</math>, where ''L'' is [[triangular matrix|lower triangular]] and ''U'' is [[triangular matrix|upper triangular]].

*Related: the [[LDU decomposition|''LDU'' decomposition]] is <math>A=LDU</math>, where ''L'' is [[triangular matrix|lower triangular]] with ones on the diagonal, ''U'' is [[triangular matrix|upper triangular]] with ones on the diagonal, and ''D'' is a [[diagonal matrix]].

*Related: the [[LUP decomposition|''LUP'' decomposition]] is <math>APA=LUPLU</math>, where ''L'' is [[triangular matrix|lower triangular]], ''U'' is [[triangular matrix|upper triangular]], and ''P'' is a [[permutation matrix]].

*Existence: An LUP decomposition exists for any square matrix ''A''. When ''P'' is an [[identity matrix]], the LUP decomposition reduces to the LU decomposition. If the LU decomposition exists, then the LDU decomposition exists.<ref>{{harvnb|Simon|Blume|1994}} Chapter 7.</ref>

*Comments: The LUP and LU decompositions are useful in solving an ''n''-by-''n'' system of linear equations <math>A \mathbf{x} = \mathbf{b}</math>. These decompositions summarize the process of [[Gaussian elimination]] in matrix form. Matrix ''P'' represents any row interchanges carried out in the process of Gaussian elimination. If Gaussian elimination produces the [[row echelon form]] without requiring any row interchanges, then ''P''&nbsp;=&nbsp;''I'', so an LU decomposition exists.

Line 35:

=== Cholesky decomposition ===

{{main|Cholesky decomposition}}

*Applicable to: [[square matrix|square]], [[symmetric matrix|hermitian]], [[positive-definite matrix|positive definite]] matrix ''<math>A''</math>

*Decomposition: <math>A=U^*U</math>, where <math>U</math> is upper triangular with real positive diagonal entries

*Comment: if the matrix <math>A</math> is Hermitian and positive semi-definite, then it has a decomposition of the form <math>A=U^*U</math> if the diagonal entries of <math>U</math> are allowed to be zero

*Uniqueness: for positive definite matrices Cholesky decomposition is unique. However, it is not unique in the positive semi-definite case.

*Comment: if <math>A</math> is real and symmetric, <math>U</math> has all real elements

*Comment: An alternative is the [[LDL decomposition]], which can avoid extracting square roots.

Line 63:

*Applicable to: [[square matrix]] ''A'' with linearly independent eigenvectors (not necessarily distinct eigenvalues).

*Decomposition: <math>A=VDV^{-1}</math>, where ''D'' is a [[diagonal matrix]] formed from the [[eigenvalue]]s of ''A'', and the columns of ''V'' are the corresponding [[eigenvector]]s of ''A''.

*Existence: An ''n''-by-''n'' matrix ''A'' always has ''n'' (complex) eigenvalues, which can be ordered (in more than one way) to form an ''n''-by-''n'' diagonal matrix ''D'' and a corresponding matrix of nonzero columns ''V'' that satisfies the [[Eigenvalue, eigenvector and eigenspace#Definitions: the eigenvalue equation|eigenvalue equation]] <math>AV=VD</math>. <math>V</math> is invertible if and only if the ''n'' eigenvectors are [[Linear independence|linearly independent]] (i.e.that is, each eigenvalue has [[geometric multiplicity]] equal to its [[algebraic multiplicity]]). A sufficient (but not necessary) condition for this to happen is that all the eigenvalues are different (in this case geometric and algebraic multiplicity are equal to 1)

*Comment: One can always normalize the eigenvectors to have length one (see the definition of the eigenvalue equation)

*Comment: Every [[normal matrix]] ''A'' (i.e.that is, matrix for which <math>AA^*=A^*A</math>, where <math>A^*</math> is a [[conjugate transpose]]) can be eigendecomposed. For a [[normal matrix]] ''A'' (and only for a normal matrix), the eigenvectors can also be made orthonormal (<math>VV^*=I</math>) and the eigendecomposition reads as <math>A=VDV^*</math>. In particular all [[Unitary matrix|unitary]], [[Hermitian matrix|Hermitian]], or [[Skew-Hermitian matrix|skew-Hermitian]] (in the real-valued case, all [[Orthogonal matrix|orthogonal]], [[Symmetric matrix|symmetric]], or [[Skew-symmetric matrix|skew-symmetric]], respectively) matrices are normal and therefore possess this property.

*Comment: For any real [[symmetric matrix]] ''A'', the eigendecomposition always exists and can be written as <math>A=VDV^\mathsf{T}</math>, where both ''D'' and ''V'' are real-valued.

*Comment: The eigendecomposition is useful for understanding the solution of a system of linear ordinary differential equations or linear difference equations. For example, the difference equation <math>x_{t+1}=Ax_t</math> starting from the initial condition <math>x_0=c</math> is solved by <math>x_t = A^tc</math>, which is equivalent to <math>x_t = VD^tV^{-1}c</math>, where ''V'' and ''D'' are the matrices formed from the eigenvectors and eigenvalues of ''A''. Since ''D'' is diagonal, raising it to power <math>D^t</math>, just involves raising each element on the diagonal to the power ''t''. This is much easier to do and understand than raising ''A'' to power ''t'', since ''A'' is usually not diagonal.

Line 96:

*Applicable to: square, complex, symmetric matrix ''A''.

*Decomposition: <math>A=VDV^\mathsf{T}</math>, where ''D'' is a real nonnegative [[diagonal matrix]], and ''V'' is [[unitary matrix|unitary]]. <math>V^\mathsf{T}</math> denotes the [[matrix transpose]] of ''V''.

*Comment: The diagonal elements of ''D'' are the nonnegative square roots of the eigenvalues of <math>AA^*=VD^2V^*{-1}</math>.

*Comment: ''V'' may be complex even if ''A'' is real.

*Comment: This is not a special case of the eigendecomposition (see above), which uses <math>V^{-1}</math> instead of <math>V^\mathsf{T}</math>. Moreover, if ''A'' is not real, it is not Hermitian and the form using <math>V^*</math> also does not apply.

Line 119:

*Comment: ''U'' and ''V'' matrices are not the same as those from the SVD.

Analogous scale-invariant decompositions can be derived from other matrix decompositions,; e.g.for example, to obtain scale-invariant eigenvalues.<ref>{{citation|last=Uhlmann |first=J.K. |title=A Generalized Matrix Inverse that is Consistent with Respect to Diagonal Transformations |journal=SIAM Journal on Matrix Analysis and Applications |year=2018 |volume=239 |issue=2 |pages=781–800 |doi=10.1137/17M113890X }}</ref><ref>{{citation|last=Uhlmann |first=J.K. |title=A Rank-Preserving Generalized Matrix Inverse for Consistency with Respect to Similarity |journal=IEEE Control Systems Letters |issn=2475-1456 |year=2018 |volume=3 |pages=91–95 |doi=10.1109/LCSYS.2018.2854240 |arxiv=1804.07334 |s2cid=5031440 }}</ref>

===Hessenberg decomposition===

*Applicable to: [[square matrix]] A.

*Decomposition: <math>A=PHP^*</math> where <math>H</math> is the [[Hessenberg matrix]] and <math>P</math> is a [[unitary matrix]].

*Comment: often the first step in the Schur decomposition.

===Complete orthogonal decomposition===

{{main|Complete orthogonal decomposition}}

*Also known as: ''UTV decomposition'', ''ULV decomposition'', ''URV decomposition''.

*Applicable to: ''m''-by-''n'' matrix ''A''.

*Decomposition: <math>A=UTV^*</math>, where ''T'' is a [[triangular matrix]], and ''U'' and ''V'' are [[unitary matrix|unitary matrices]].

*Comment: Similar to the singular value decomposition and to the Schur decomposition.

== Other decompositions ==

Line 133 ⟶ 145:

*Applicable to: square, complex, non-singular matrix ''A''.<ref>{{harvnb|Choudhury|Horn|1987|pp=219–225}}</ref>

*Decomposition: <math>A=QS</math>, where ''Q'' is a complex orthogonal matrix and ''S'' is complex symmetric matrix.

*Uniqueness: If <math>A^\mathsf{T}A</math> has no negative real eigenvalues, then the decomposition is unique.<ref name=":0">{{Cite journal|last=Bhatia|first=Rajendra|date=2013-11-15|title=The bipolar decomposition|journal=Linear Algebra and Its Applications|volume=439|issue=10|pages=3031–3037|doi=10.1016/j.laa.2013.09.006|doi-access=free}}</ref>

*Comment: The existence of this decomposition is equivalent to <math>AA^\mathsf{T}</math> being similar to <math>A^\mathsf{T}A</math>.<ref>{{harvnb|Horn|Merino|1995|pp=43–92}}</ref>

*Comment: A variant of this decomposition is <math>A=RC</math>, where ''R'' is a real matrix and ''C'' is a [[circular matrix]].<ref name=":0" />

=== Mostow's decomposition ===

* Applicable to: square, complex, non-singular matrix ''A''.<ref>{{citation|last=Mostow|first= G. D.|title= Some new decomposition theorems for semi-simple groups|series= Mem. Amer. Math. Soc. |year=1955|volume=14|pages= 31–54|url=https://archive.org/details/liealgebrasandli029541mbp|publisher= American Mathematical Society}}</ref><ref>{{Cite book|title=Matrix Information Geometry|lastlast1=Nielsen|firstfirst1=Frank|last2=Bhatia|first2=Rajendra|publisher=Springer|year=2012|isbn=9783642302329|pages=224|language=en|doi=10.1007/978-3-642-30232-9|arxiv = 1007.4402|s2cid=118466496 }}</ref>

* Decomposition: <math>A=Ue^{iM}e^{S}</math>, where ''U'' is unitary, ''M'' is real anti-symmetric and ''S'' is real symmetric.

* Comment: The matrix ''A'' can also be decomposed as <math>A=U_2e^{S_2}e^{iM_2}</math>, where ''U''₂ is unitary, ''M''₂ is real anti-symmetric and ''S''₂ is real symmetric.<ref name=":0" />

Line 149 ⟶ 161:

=== Sectoral decomposition ===

*Applicable to: square, complex matrix ''A'' with [[numerical range]] contained in the sector <math>S_\alpha = \left\{r e^{i \theta} \in \mathbb{C} \mid r> 0, |\theta| \le \alpha < \frac{\pi}{2}\right\}</math>.

*Decomposition: <math>A = CZC^*</math>, where ''C'' is an invertible complex matrix and <math>Z = \operatorname{diag}\left(e^{i\theta_1},\ldots,e^{i\theta_n}\right)</math> with all <math>\left|\theta_j\right| \le \alpha </math>.<ref name=Zhang2014>{{cite journal|last1=Zhang|first1=Fuzhen|title=A matrix decomposition and its applications|journal=Linear and Multilinear Algebra|volume=63|issue=10|date=30 June 2014|pages=2033–2042|doi=10.1080/03081087.2014.933219|s2cid=19437967 |url=https://zenodo.org/record/851661/files/article.pdf}}</ref><ref>{{cite journal|last1=Drury|first1=S.W.|title=Fischer determinantal inequalities and Highamʼs Conjecture|journal=Linear Algebra and Its Applications|date=November 2013|volume=439|issue=10|pages=3129–3133|doi=10.1016/j.laa.2013.08.031|doi-access=free}}</ref>

=== Williamson's normal form ===

* Applicable to: square, [[Positive-definite matrix|positive-definite]] real matrix ''A'' with order 2''n''×2''n''.

* Decomposition: <math>A=S^\mathsf{T}\operatorname{diag}(D,D)S</math>, where <math>S \in \text{Sp}(2n)</math> is a [[symplectic matrix]] and ''D'' is a nonnegative ''n''-by-''n'' diagonal matrix.<ref>{{Cite journal|lastlast1=Idel|firstfirst1=Martin|last2=Soto Gaona|first2=Sebastián|last3=Wolf|first3=Michael M.|date=2017-07-15|title=Perturbation bounds for Williamson's symplectic normal form|journal=Linear Algebra and Its Applications|volume=525|pages=45–58|doi=10.1016/j.laa.2017.03.013|arxiv=1609.01338|s2cid=119578994 }}</ref>

===Matrix square root===

Line 185 ⟶ 197:

*{{Citation|last1=Meyer|first1=C. D.|title=Matrix Analysis and Applied Linear Algebra|url=http://www.matrixanalysis.com/|publisher=[[Society for Industrial and Applied Mathematics|SIAM]]|isbn=978-0-89871-454-8|year=2000}}

*{{citation|first=E.|last=Schmidt|title=Zur Theorie der linearen und nichtlinearen Integralgleichungen. I Teil. Entwicklung willkürlichen Funktionen nach System vorgeschriebener|journal=Mathematische Annalen|volume=63|issue=4|pages=433–476|year=1907|language=de|author-link=Erhard Schmidt|doi=10.1007/bf01449770|url=https://zenodo.org/record/1428258}}

*{{Cite book|lastlast1=Simon|firstfirst1=C.|last2=Blume|first2=L.|year=1994|title=Mathematics for Economists|publisher= Norton|isbn=978-0-393-95733-4}}

*{{citation|last=Stewart|first=G. W.|year=2011|title=Fredholm, Hilbert, Schmidt: three fundamental papers on integral equations|url=http://www.cs.umd.edu/~stewart/FHS.pdf|access-date=2015-01-06}}

*{{citation|lastlast1=Townsend|firstfirst1=A.|last2=Trefethen|first2=L. N.|year=2015|title=Continuous analogues of matrix factorizations|journal=[[Proceedings of the Royal Society|Proc. R. Soc. A]]|volume=471|issue=2173|pages=20140585|doi=10.1098/rspa.2014.0585|pmid=25568618|pmc=4277194|bibcode=2014RSPSA.47140585T}}

*{{citation|last=Jun|first=Lu|year=2021|title=Numerical matrix decomposition and its modern applications: A rigorous first course|url=https://arxiv.org/abs/2107.02579|arxiv=2107.02579|access-date=2021-11-17}}