Projection matrix: Difference between revisions

Revision as of 15:36, 3 March 2019

Revision as of 16:04, 3 March 2019

Line 44:

: <math>

\hat{\mathbf\beta}_{\text{GLS}}= \left( \mathbf{X}^{\topmathsf{T} \mathbf{\Psi}^{-1} \mathbf{X} \right)^{-1} \mathbf{X}^{\topmathsf{T} \mathbf{\Psi}^{-1}\mathbf{y}

</math>.

Line 50:

: <math>

H = \mathbf{X}\left( \mathbf{X}^{\topmathsf{T} \mathbf{\Psi}^{-1} \mathbf{X} \right)^{-1} \mathbf{X}^{\topmathsf{T} \mathbf{\Psi}^{-1}

</math>

Line 60:

* <math>\mathbf{P}</math> is symmetric, and so is <math>\mathbf{M} \equiv \left( \mathbf{I} - \mathbf{P} \right)</math>.

* <math>\mathbf{P}</math> is idempotent: <math>\mathbf{P}^2 = \mathbf{P}</math>, and so is <math>\mathbf{M}</math>.

* If <math>\mathbf{X}</math> is an ({{nowrap|''n'' × ''r'')}} matrix with <math>\operatorname{rank}(\mathbf{X})=r</math>, then <math>\operatorname{rank}(\mathbf{P}) = r</math>

* The [[eigenvalue]]s of <math>\mathbf{P}</math> consist of ''r'' ones and {{nowrap|''n−rn'' − ''r''}} zeros, while the eigenvalues of <math>\mathbf{M}</math> consist of {{nowrap|''n−rn'' − ''r''}} ones and ''r'' zeros.<ref>{{cite book |first=Takeshi |last=Amemiya |title=Advanced Econometrics |___location=Cambridge |publisher=Harvard University Press |year=1985 |isbn=0-674-00560-0 |pages=460–461 |url=https://books.google.com/books?id=0bzGQE14CwEC&pg=PA460 }}</ref>

* <math>\mathbf{X}</math> is invariant under <math>\mathbf{P}</math> : <math>\mathbf{P X} = \mathbf{X},</math> hence <math>\left( \mathbf{I} - \mathbf{P} \right) \mathbf{X} = \mathbf{0}</math>.

* <math>\left( \mathbf{I} - \mathbf{P} \right) \mathbf{P} = \mathbf{P} \left( \mathbf{I} - \mathbf{P} \right) = \mathbf{0}.</math>

Line 74:

Suppose the design matrix <math>X</math> can be decomposed by columns as <math>X= [A~~~B]</math>.

Define the hat or projection operator as <math>P\{X\} = X \left(X^\topmathsf{T} X \right)^{-1} X^\topmathsf{T}</math>. Similarly, define the residual operator as <math>M\{X\} = I - P\{X\}</math>.

Then the projection matrix can be decomposed as follows:<ref>{{cite book|last1=Rao|first1=C. Radhakrishna|last2=Toutenburg|first2=Helge|author3=Shalabh|first4=Christian|last4=Heumann|title=Linear Models and Generalizations|year=2008|publisher=Springer|___location=Berlin|isbn=978-3-540-74226-5|pages=323|edition=3rd}}</ref>

:<math>

P\{X\} = P\{A\} + P\{M\{A\} B\},

</math>

where, e.g., <math>P\{A\} = A \left(A^\topmathsf{T} A \right)^{-1} A^\topmathsf{T}</math> and <math>M\{A\} = I - P\{A\}</math>.

There are a number of applications of such a decomposition. In the classical application <math>A</math> is a column of all ones, which allows one to analyze the effects of adding an intercept term to a regression. Another use is in the [[fixed effects model]], where <math>A</math> is a large [[sparse matrix]] of the dummy variables for the fixed effect terms. One can use this partition to compute the hat matrix of <math>X </math> without explicitly forming the matrix <math>X</math>, which might be too large to fit into computer memory.