Content deleted Content added
Takagi's factorization: V^* form also doesn't apply |
Citation bot (talk | contribs) Removed URL that duplicated identifier. | Use this bot. Report bugs. | #UCB_CommandLine |
||
(47 intermediate revisions by 31 users not shown) | |||
Line 1:
{{Distinguish|
{{Short description|Representation of a matrix as a product}} In the [[mathematics|mathematical]] discipline of [[linear algebra]], a '''matrix decomposition''' or '''matrix factorization''' is a [[factorization]] of a [[Matrix (mathematics)|matrix]] into a product of matrices. There are many different matrix decompositions; each finds use among a particular class of problems.
Line 5 ⟶ 6:
In [[numerical analysis]], different decompositions are used to implement efficient matrix [[algorithm]]s.
For
Similarly, the [[QR decomposition]] expresses ''A'' as ''QR'' with ''Q'' an [[orthogonal matrix]] and ''R'' an upper triangular matrix.
== Decompositions related to solving systems of linear equations ==
Line 13 ⟶ 14:
=== LU decomposition ===
{{main|LU decomposition}}
*Traditionally applicable to: [[square matrix]] ''A'', although rectangular matrices can be applicable.<ref>{{Cite book|last=Lay|first=David C.|title=Linear algebra and its applications|date=2016|others=Steven R. Lay, Judith McDonald|isbn=978-1-292-09223-2|edition=Fifth Global|___location=Harlow|pages=142|oclc=920463015}}</ref><ref group="nb">If a non-square matrix is used, however, then the matrix ''U'' will also have the same rectangular shape as the original matrix ''A''. And so, calling the matrix ''U'' upper triangular would be incorrect as the correct term would be that ''U'' is the 'row echelon form' of ''A''. Other than this, there are no differences in LU factorization for square and non-square matrices.</ref>
*Applicable to: [[square matrix]] ''A''▼
*Decomposition: <math>A=LU</math>, where ''L'' is [[triangular matrix|lower triangular]] and ''U'' is [[triangular matrix|upper triangular]].
*Related: the [[LDU decomposition|''LDU'' decomposition]] is <math>A=LDU</math>, where ''L'' is [[triangular matrix|lower triangular]] with ones on the diagonal, ''U'' is [[triangular matrix|upper triangular]] with ones on the diagonal, and ''D'' is a [[diagonal matrix]].
*Related: the [[LUP decomposition|''LUP'' decomposition]] is <math>
*Existence: An LUP decomposition exists for any square matrix ''A''. When ''P'' is an [[identity matrix]], the LUP decomposition reduces to the LU decomposition.
*Comments: The LUP and LU decompositions are useful in solving an ''n''-by-''n'' system of linear equations <math>
=== LU reduction ===
Line 30 ⟶ 31:
*Applicable to: ''m''-by-''n'' matrix ''A'' of rank ''r''
*Decomposition: <math>A=CF</math> where ''C'' is an ''m''-by-''r'' full column rank matrix and ''F'' is an ''r''-by-''n'' full row rank matrix
*Comment: The rank factorization can be used to [[Moore–Penrose pseudoinverse#Rank decomposition|compute the Moore–Penrose pseudoinverse]] of ''A'',<ref>{{cite journal|last1=Piziak|first1=R.|last2=Odell|first2=P. L.|title=Full Rank Factorization of Matrices|journal=Mathematics Magazine|date=1 June 1999|volume=72|issue=3|pages=193|doi=10.2307/2690882|jstor=2690882}}</ref> which one can apply to [[Moore–Penrose pseudoinverse#Obtaining all solutions of a linear system|obtain all solutions of the linear system]] <math>
=== Cholesky decomposition ===
{{main|Cholesky decomposition}}
*Applicable to: [[square matrix|square]], [[symmetric matrix|hermitian]], [[positive-definite matrix|positive definite]] matrix
*Decomposition: <math>A=U^*U</math>, where
*Comment: if the matrix
*Uniqueness: for positive definite matrices Cholesky decomposition is unique. However, it is not unique in the positive semi-definite case.
*Comment: if <math>A</math> is real and symmetric, <math>U</math> has all real elements
*Comment: An alternative is the [[LDL decomposition]], which can avoid extracting square roots.
Line 44 ⟶ 45:
{{main|QR decomposition}}
*Applicable to: ''m''-by-''n'' matrix ''A'' with linearly independent columns
*Decomposition: <math>A=QR</math> where
*Uniqueness: In general it is not unique, but if <math>A</math> is of full [[Matrix rank|rank]], then there exists a single <math>R</math> that has all positive diagonal elements. If <math>A</math> is square, also <math>Q</math> is unique.
*Comment: The QR decomposition provides an
=== RRQR factorization ===
Line 62 ⟶ 63:
*Applicable to: [[square matrix]] ''A'' with linearly independent eigenvectors (not necessarily distinct eigenvalues).
*Decomposition: <math>A=VDV^{-1}</math>, where ''D'' is a [[diagonal matrix]] formed from the [[eigenvalue]]s of ''A'', and the columns of ''V'' are the corresponding [[eigenvector]]s of ''A''.
*Existence: An ''n''-by-''n'' matrix ''A'' always has ''n'' (complex) eigenvalues, which can be ordered (in more than one way) to form an ''n''-by-''n'' diagonal matrix ''D'' and a corresponding matrix of nonzero columns ''V'' that satisfies the [[
*Comment: One can always normalize the eigenvectors to have length one (see the definition of the eigenvalue equation)
*Comment: Every [[normal matrix]] ''A'' (
*Comment: For any real [[symmetric matrix]] ''A'', the eigendecomposition always exists and can be written as <math>A=VDV^\mathsf{T}</math>, where both ''D'' and ''V'' are real-valued.
*Comment: The eigendecomposition is useful for understanding the solution of a system of linear ordinary differential equations or linear difference equations. For example, the difference equation <math>x_{t+1}=Ax_t</math> starting from the initial condition <math>x_0=c</math> is solved by <math>x_t = A^tc</math>, which is equivalent to <math>x_t = VD^tV^{-1}c</math>, where ''V'' and ''D'' are the matrices formed from the eigenvectors and eigenvalues of ''A''. Since ''D'' is diagonal, raising it to power <math>D^t</math>, just involves raising each element on the diagonal to the power ''t''. This is much easier to do and understand than raising ''A'' to power ''t'', since ''A'' is usually not diagonal.
Line 77 ⟶ 78:
*Applicable to: [[square matrix]] ''A''
*Decomposition (complex version): <math>A=UTU^*</math>, where ''U'' is a [[unitary matrix]], <math>U^*</math> is the [[conjugate transpose]] of ''U'', and ''T'' is an [[upper triangular]] matrix called the complex [[Schur form]] which has the [[eigenvalue]]s of ''A'' along its diagonal.
*Comment: if ''A'' is a [[normal matrix]], then ''T'' is diagonal and the Schur decomposition coincides with the spectral decomposition.
=== Real Schur decomposition ===
*Applicable to: [[square matrix]] ''A''
*Decomposition: This is a version of Schur decomposition where <math>V</math> and <math>S</math> only contain real numbers. One can always write <math>A=VSV^\mathsf{T}</math> where ''V'' is a real [[orthogonal matrix]], <math>V^\mathsf{T}</math> is the [[matrix transpose|transpose]] of ''V'', and ''S'' is a [[block matrix|block upper triangular]] matrix called the real [[Schur form]]. The blocks on the diagonal of ''S'' are of size 1×1 (in which case they represent real eigenvalues) or 2×2 (in which case they are derived from [[complex conjugate]] eigenvalue pairs).
=== QZ decomposition ===
Line 89 ⟶ 90:
*Comment: there are two versions of this decomposition: complex and real.
*Decomposition (complex version): <math>A=QSZ^*</math> and <math>B=QTZ^*</math> where ''Q'' and ''Z'' are [[unitary matrix|unitary matrices]], the * superscript represents [[conjugate transpose]], and ''S'' and ''T'' are [[upper triangular]] matrices.
*Comment: in the complex QZ decomposition, the ratios of the diagonal elements of ''S'' to the corresponding diagonal elements of ''T'', <math>\lambda_i = S_{ii}/T_{ii}</math>, are the generalized [[eigenvalue]]s that solve the [[Eigendecomposition of a matrix#Additional topics|generalized eigenvalue problem]] <math>
*Decomposition (real version): <math>A=QSZ^\mathsf{T}</math> and <math>B=QTZ^\mathsf{T}</math> where ''A'', ''B'', ''Q'', ''Z'', ''S'', and ''T'' are matrices containing real numbers only. In this case ''Q'' and ''Z'' are [[orthogonal matrix|orthogonal matrices]], the ''T'' superscript represents [[matrix transpose|transposition]], and ''S'' and ''T'' are [[block matrix|block upper triangular]] matrices. The blocks on the diagonal of ''S'' and ''T'' are of size 1×1 or 2×2.
=== Takagi's factorization ===
*Applicable to: square, complex, symmetric matrix ''A''.
*Decomposition: <math>A=VDV^\mathsf{T}</math>, where ''D'' is a real nonnegative [[diagonal matrix]], and ''V'' is [[unitary matrix|unitary]]. <math>V^\mathsf{T}</math> denotes the [[matrix transpose]] of ''V''.
*Comment: The diagonal elements of ''D'' are the nonnegative square roots of the eigenvalues of <math>AA^*=VD^2V^{-1}</math>.
*Comment: ''V'' may be complex even if ''A'' is real.
*Comment: This is not a special case of the eigendecomposition (see above), which uses <math>V^{-1}</math> instead of <math>V^\mathsf{T}</math>. Moreover, if ''A'' is not real, it is not Hermitian and the form using <math>V^*</math> also does not apply.
=== Singular value decomposition ===
{{main|Singular value decomposition}}
*Applicable to: ''m''-by-''n'' matrix ''A''.
*Decomposition: <math>A=UDV^*</math>, where ''D'' is a nonnegative [[diagonal matrix]], and
*Comment: The diagonal elements of ''D'' are called the [[singular value]]s of ''A''.
*Comment: Like the eigendecomposition above, the singular value decomposition involves finding basis directions along which matrix multiplication is equivalent to scalar multiplication, but it has greater generality since the matrix under consideration need not be square.
Line 118 ⟶ 119:
*Comment: ''U'' and ''V'' matrices are not the same as those from the SVD.
Analogous scale-invariant decompositions can be derived from other matrix decompositions
*Decomposition: <math>A=PHP^*</math> where <math>H</math> is the [[Hessenberg matrix]] and <math>P</math> is a [[unitary matrix]].
*Comment: often the first step in the Schur decomposition.
===Complete orthogonal decomposition===
{{main|Complete orthogonal decomposition}}
*Also known as: ''UTV decomposition'', ''ULV decomposition'', ''URV decomposition''.
*Applicable to: ''m''-by-''n'' matrix ''A''.
*Decomposition: <math>A=UTV^*</math>, where ''T'' is a [[triangular matrix]], and ''U'' and ''V'' are [[unitary matrix|unitary matrices]].
*Comment: Similar to the singular value decomposition and to the Schur decomposition.
== Other decompositions ==
Line 132 ⟶ 145:
*Applicable to: square, complex, non-singular matrix ''A''.<ref>{{harvnb|Choudhury|Horn|1987|pp=219–225}}</ref>
*Decomposition: <math>A=QS</math>, where ''Q'' is a complex orthogonal matrix and ''S'' is complex symmetric matrix.
*Uniqueness: If <math>A^\mathsf{T}A</math> has no negative real eigenvalues, then the decomposition is unique.<ref name=":0">{{Cite journal|last=Bhatia|first=Rajendra|date=2013-11-15|title=The bipolar decomposition|journal=Linear Algebra and Its Applications|volume=439|issue=10|pages=3031–3037|doi=10.1016/j.laa.2013.09.006|doi-access=
*Comment: The existence of this decomposition is equivalent to <math>AA^\mathsf{T}</math> being similar to <math>A^\mathsf{T}A</math>.<ref>{{harvnb|Horn|
*Comment: A variant of this decomposition is <math>A=RC</math>, where ''R'' is a real matrix and ''C'' is a [[circular matrix]].<ref name=":0" />
=== Mostow's decomposition ===
* Applicable to: square, complex, non-singular matrix ''A''.<ref>{{citation|last=Mostow|first= G. D.|title= Some new decomposition theorems for semi-simple groups|series= Mem. Amer. Math. Soc. |year=1955|volume=14|pages= 31–54|url=https://archive.org/details/liealgebrasandli029541mbp|publisher= American Mathematical Society}}</ref><ref>{{Cite book|title=Matrix Information Geometry|
▲{{main|Mostow decomposition}}
▲* Applicable to: square, complex, non-singular matrix ''A''.<ref>{{citation|last=Mostow|first= G. D.|title= Some new decomposition theorems for semi-simple groups|series= Mem. Amer. Math. Soc. |year=1955|volume=14|pages= 31–54|url=https://archive.org/details/liealgebrasandli029541mbp|publisher= American Mathematical Society}}</ref><ref>{{Cite book|title=Matrix Information Geometry|last=Nielsen|first=Frank|last2=Bhatia|first2=Rajendra|publisher=Springer|year=2012|isbn=9783642302329|pages=224|language=en|doi=10.1007/978-3-642-30232-9|arxiv = 1007.4402}}</ref>
* Decomposition: <math>A=Ue^{iM}e^{S}</math>, where ''U'' is unitary, ''M'' is real anti-symmetric and ''S'' is real symmetric.
* Comment: The matrix ''A'' can also be decomposed as <math>A=U_2e^{S_2}e^{iM_2}</math>, where ''U''<sub>2</sub>
=== Sinkhorn normal form ===
Line 149 ⟶ 161:
=== Sectoral decomposition ===
*Applicable to: square, complex matrix ''A'' with [[numerical range]] contained in the sector <math>S_\alpha = \left\{r e^{i \theta} \in \mathbb{C} \mid r> 0, |\theta| \le \alpha < \frac{\pi}{2}\right\}</math>.
*Decomposition: <math>A = CZC^*</math>, where ''C'' is an invertible complex matrix and <math>Z = \operatorname{diag}\left(e^{i\theta_1},\ldots,e^{i\theta_n}\right)</math> with all <math>\left|\theta_j\right| \le \alpha </math>.<ref name=Zhang2014>{{cite journal|last1=Zhang|first1=Fuzhen|title=A matrix decomposition and its applications|journal=Linear and Multilinear Algebra|volume=63|issue=10|date=30 June 2014|pages=2033–2042|doi=10.1080/03081087.2014.933219|s2cid=19437967 |url=https://zenodo.org/record/851661
=== Williamson's normal form ===
* Applicable to: square, [[Positive-definite matrix|positive-definite]] real matrix ''A'' with order 2''n''
* Decomposition: <math>A=S^\mathsf{T}\
===Matrix square root===
{{main|Square root of a matrix}}
* Decomposition: <math>A=BB</math>, not unique in general.
* In the case of positive semidefinite <math>A</math>, there is a unique positive semidefinite <math>B</math> such that <math>A=B^*B=BB</math>.
== Generalizations ==
Line 161 ⟶ 178:
These factorizations are based on early work by {{harvtxt|Fredholm|1903}}, {{harvtxt|Hilbert|1904}} and {{harvtxt|Schmidt|1907}}. For an account, and a translation to English of the seminal papers, see {{harvtxt|Stewart|2011}}.
== See also ==
* [[Matrix splitting]]
* [[Non-negative matrix factorization]]
* [[Principal component analysis]]
==
===Notes===
{{reflist|group=nb}}
===Citations===
{{reflist}}
==
*{{cite journal|last1=Choudhury|first1=Dipa|last2=Horn|first2=Roger A.|title=A Complex Orthogonal-Symmetric Analog of the Polar Decomposition|journal=SIAM Journal on Algebraic and Discrete Methods|date=April 1987|volume=8|issue=2|pages=219–225|doi=10.1137/0608019}}
*{{citation|first=I.|last=Fredholm|title=Sur une classe d'´equations fonctionnelles|journal=Acta Mathematica|volume=27|pages=365–390|year=1903|language=fr|author-link=Ivar Fredholm|doi=10.1007/bf02421317|doi-access=free}}
Line 176 ⟶ 197:
*{{Citation|last1=Meyer|first1=C. D.|title=Matrix Analysis and Applied Linear Algebra|url=http://www.matrixanalysis.com/|publisher=[[Society for Industrial and Applied Mathematics|SIAM]]|isbn=978-0-89871-454-8|year=2000}}
*{{citation|first=E.|last=Schmidt|title=Zur Theorie der linearen und nichtlinearen Integralgleichungen. I Teil. Entwicklung willkürlichen Funktionen nach System vorgeschriebener|journal=Mathematische Annalen|volume=63|issue=4|pages=433–476|year=1907|language=de|author-link=Erhard Schmidt|doi=10.1007/bf01449770|url=https://zenodo.org/record/1428258}}
*{{Cite book|
*{{citation|last=Stewart|first=G. W.|year=2011|title=Fredholm, Hilbert, Schmidt: three fundamental papers on integral equations|url=http://www.cs.umd.edu/~stewart/FHS.pdf|access-date=2015-01-06}}
*{{citation|
*{{citation|last=Jun|first=Lu|year=2021|title=Numerical matrix decomposition and its modern applications: A rigorous first course|arxiv=2107.02579}}
== External links ==
*[http://www.bluebit.gr/matrix-calculator/ Online Matrix Calculator]
*[http://www.wolframalpha.com/input/?i=matrix+decomposition&rawformassumption={%22C%22,+%22matrix+decomposition%22}+-%3E+{%22Calculator%22}&rawformassumption={%22MC%22,%22%22}-%3E{%22Formula%22} Wolfram Alpha Matrix Decomposition Computation » LU and QR Decomposition]
*[
*
{{linear algebra}}
Line 190 ⟶ 213:
[[Category:Matrix theory]]
[[Category:Matrix decompositions| ]]
[[Category:factorization]]
|