Projection matrix

The hat matrix, H, is used in statistics to relate errors in residuals to experimental errors. Suppose that a linear least squares problem is being addressed. The model can be written as

${\displaystyle \mathbf {y^{calc}=Jp} }$

where J is a matrix of coefficients and p is a vector of parameters. The solution to the un-weighted least-squares equations is given by

${\displaystyle \mathbf {p=\left(J^{T}J\right)^{-1}J^{T}y^{obs}} }$

The vector of un-weighted residuals, r, is given by

${\displaystyle \mathbf {r=y^{obs}-y^{calc}=y^{obs}-J\left(J^{T}J\right)^{-1}J^{T}y^{obs}} }$

The matrix ${\displaystyle \mathbf {J\left(J^{T}J\right)^{-1}J^{T}} }$ is known as the hat matrix. Thus, the residuals can be expressed simply as

${\displaystyle \mathbf {r=\left(I-H\right)y^{obs}} }$

The variance-covariance matrix of the residuals is, by error propagation, equal to ${\displaystyle \mathbf {\left(I-H\right)M\left(I-H\right)} }$, where M is the variance-covariance matrix of the observations.

The hat matrix is idempotent, that is, ${\displaystyle \mathbf {HH=H} }$. Some other useful properties of the hat matrix are summarized in ^[1]

References

1. P. Gans, Data Fitting in the Chemical Sciences,, Wiley, 1992.