Projection matrix: Difference between revisions

Revision as of 16:34, 17 March 2021

Revision as of 07:16, 4 April 2021

Line 86:

== Blockwise formula ==

Suppose the design matrix <math>X</math> can be decomposed by columns as <math>X = [\begin{bmatrix} A~~~ & B] \end{bmatrix}</math>.

Define the hat or projection operator as <math>P\{X\} = X \left(X^\textsf{T} X \right)^{-1} X^\textsf{T}</math>. Similarly, define the residual operator as <math>M\{X\} = I - P\{X\}</math>.

Then the projection matrix can be decomposed as follows:<ref>{{cite book|last1=Rao|first1=C. Radhakrishna|last2=Toutenburg|first2=Helge|author3=Shalabh|first4=Christian|last4=Heumann|title=Linear Models and Generalizations|url=https://archive.org/details/linearmodelsgene00raop|url-access=limited|year=2008|publisher=Springer|___location=Berlin|isbn=978-3-540-74226-5|pages=[https://archive.org/details/linearmodelsgene00raop/page/n335 323]|edition=3rd}}</ref>

:<math> P\{X\} = P\{A\} + P\{M\{A\} B\}, </math>

:<math>

P\{X\} = P\{A\} + P\{M\{A\} B\},

</math>

where, e.g., <math>P\{A\} = A \left(A^\textsf{T} A \right)^{-1} A^\textsf{T}</math> and <math>M\{A\} = I - P\{A\}</math>.

There are a number of applications of such a decomposition. In the classical application <math>A</math> is a column of all ones, which allows one to analyze the effects of adding an intercept term to a regression. Another use is in the [[fixed effects model]], where <math>A</math> is a large [[sparse matrix]] of the dummy variables for the fixed effect terms. One can use this partition to compute the hat matrix of <math>X </math> without explicitly forming the matrix <math>X</math>, which might be too large to fit into computer memory.