Projection matrix: Difference between revisions

Revision as of 12:29, 13 March 2023

Revision as of 02:38, 1 August 2023

Line 92:

== Blockwise formula ==

Suppose the design matrix <math>\mathbf{X}</math> can be decomposed by columns as <math>\mathbf{X} = \begin{bmatrix} \mathbf{A} & \mathbf{B} \end{bmatrix}</math>.

Define the hat or projection operator as <math>\mathbf{P}[\mathbf{X\}] := \mathbf{X} \left(\mathbf{X}^\textsf{T} \mathbf{X} \right)^{-1} \mathbf{X}^\textsf{T}</math>. Similarly, define the residual operator as <math>\mathbf{M}[\mathbf{X\}] := \mathbf{I} - \mathbf{P}[\mathbf{X\}]</math>.

Then the projection matrix can be decomposed as follows:<ref>{{cite book|last1=Rao|first1=C. Radhakrishna|last2=Toutenburg|first2=Helge|author3=Shalabh|first4=Christian|last4=Heumann|title=Linear Models and Generalizations|url=https://archive.org/details/linearmodelsgene00raop|url-access=limited|year=2008|publisher=Springer|___location=Berlin|isbn=978-3-540-74226-5|pages=[https://archive.org/details/linearmodelsgene00raop/page/n335 323]|edition=3rd}}</ref>

:<math> \mathbf{P}[\mathbf{X\}] = \mathbf{P}[\mathbf{A\}] + \mathbf{P}\big[\mathbf{M}[\mathbf{A\}] B\mathbf{B}\big], </math>

where, e.g., <math>\mathbf{P}[\mathbf{A\}] = \mathbf{A} \left(\mathbf{A}^\textsf{T} \mathbf{A} \right)^{-1} \mathbf{A}^\textsf{T}</math> and <math>\mathbf{M}[\mathbf{A\}] = \mathbf{I} - \mathbf{P}[\mathbf{A\}]</math>.

There are a number of applications of such a decomposition. In the classical application <math>\mathbf{A}</math> is a column of all ones, which allows one to analyze the effects of adding an intercept term to a regression. Another use is in the [[fixed effects model]], where <math>\mathbf{A}</math> is a large [[sparse matrix]] of the dummy variables for the fixed effect terms. One can use this partition to compute the hat matrix of <math>\mathbf{X }</math> without explicitly forming the matrix <math>\mathbf{X}</math>, which might be too large to fit into computer memory.

== See also ==