Projection matrix: Difference between revisions

Revision as of 09:55, 13 April 2020

Revision as of 03:45, 18 April 2020

Line 5:

If the vector of [[Response variable|response values]] is denoted by <math>\mathbf{y}</math> and the vector of fitted values by <math>\mathbf{\hat{y}}</math>,

:<math>\mathbf{\hat{y}} = \mathbf{P} \mathbf{y}.</math>

As <math>\mathbf{\hat{y}}</math> is usually pronounced "y-hat", the projection matrix is also named ''hat matrix'' as it "puts a [[circumflex|hat]] on <math>\mathbf{y}</math>". The formula for the vector of [[errors and residuals in statistics|residual]]s <math>\mathbf{ur}</math> can also be expressed compactly using the projection matrix:

:<math>\mathbf{ur} = \mathbf{y} - \mathbf{\hat{y}} = \mathbf{y} - \mathbf{P} \mathbf{y} = \left( \mathbf{I} - \mathbf{P} \right) \mathbf{y}.</math>

where <math>\mathbf{I}</math> is the [[identity matrix]]. The matrix <math>\mathbf{M} \equiv \left( \mathbf{I} - \mathbf{P} \right)</math> is sometimes referred to as the '''residual maker matrix'''. Moreover, the element in the ''i''th row and ''j''th column of <math>\mathbf{P}</math> is equal to the [[covariance]] between the ''j''th response value and the ''i''th fitted value, divided by the [[variance]] of the former:

:<math>

Line 13:

\end{align}

</math>

Therefore, the [[covariance matrix]] of the residuals <math>\mathbf{ur}</math>, by [[error propagation]], equals

:<math>\mathbf{\Sigma}_\mathbf{ur} = \left( \mathbf{I}-\mathbf{P} \right)^{\mathsf{T}} \mathbf{\Sigma} \left( \mathbf{I}-\mathbf{P} \right)</math>,

where <math>\mathbf{\Sigma}</matH> is the [[covariance matrix]] of the error vector (and by extension, the response vector as well). For the case of linear models with [[independent and identically distributed]] errors in which <math>\mathbf{\Sigma} = \sigma^{2} \mathbf{I}</math>, this reduces to:<ref name="Hoaglin1977"/>

:<math>\mathbf{\Sigma}_\mathbf{ur} = \left( \mathbf{I} - \mathbf{P} \right) \sigma^{2}</math>.

== Linear model ==