Information matrix test: Difference between revisions

Revision as of 18:53, 1 September 2017

Revision as of 16:11, 2 September 2017

Line 1:

In [[econometrics]], the '''information matrix test''' is used to determine whether a [[regression model]] is [[Specification (regression)|misspecified]]. The test was developed by [[Halbert White]],<ref>{{Cite journal |last1=White|first1=Halbert|title=Maximum Likelihood Estimation of Misspecified Models |journal=[[Econometrica]] |date=1982 |volume=50 |issue=1 |pages=1–25 |jstor=1912526 }}</ref> who observed that in a correctly specified model and under standard regularity assumptions, the [[Fisher information|information matrix]] can be expressed in either of two ways: as the [[outer product]] of the [[gradient]], or as a function of the [[Hessian matrix]] of the log-likelihood function.

Consider a linear model <math>\mathbf{y} = \mathbf{X} \mathbf{\beta} + \mathbf{u}</math>, where the errors <math>\mathbf{u}</math> are assumed to be distributed <math>\mathrm{N} \left( 0, \sigma^{2} \mathbf{I} \right)</math>. If the parameters <math>\beta</math> and <math>\sigma^{2}</math> are stacked in the vector <math>\mathbf{\theta}^{\mathsf{T}} = \begin{bmatrix} \beta & \sigma^{2} \end{bmatrix}</math>, the resulting [[Likelihood function|log-likelihood function]] is

:<math>\mathcal{l}ell \left( \mathbf{\theta} \right) = - \frac{n}{2} \log \sigma^{2} - \frac{1}{2 \sigma^{2}} \left( \mathbf{y} - \mathbf{X} \mathbf{\beta} \right)^{\mathsf{T}} \left( \mathbf{y} - \mathbf{X} \mathbf{\beta} \right)</math>

The information matrix can then be expressed as

:<math>\mathbf{I} \left( \mathbf{\theta} \right) = \mathbb{E} \left[ \left( \frac{\partial \mathcal{l}ell \left( \mathbf{\theta} \right) }{ \partial \mathbf{\theta} } \right) \left( \frac{\partial \mathcal{l}ell \left( \mathbf{\theta} \right) }{ \partial \mathbf{\theta} } \right)^{\mathsf{T}} \right]</math>

that is the expected value of the outer product of the gradient or [[Score (statistics)|score]]. Second, it can be written as the negative of the Hessian matrix of the log-likelihood function

:<math>\mathbf{I} \left( \mathbf{\theta} \right) = - \mathbb{E} \left[ \frac{\partial^{2} \mathcal{l}ell \left( \mathbf{\theta} \right) }{ \partial \mathbf{\theta} \, \partial \mathbf{\theta}^{\mathsf{T}} } \right]</math>

If the model is correctly specified, both expressions should be equal. Combining the equivalent forms yields

:<math>\mathbf{\Delta} \left( \mathbf{\theta} \right) = \sum_{i=1}^{n} \left[ \frac{\partial^{2} \mathcal{l} \left( \mathbf{\theta} \right) }{ \partial \mathbf{\theta} \, \partial \mathbf{\theta}^{\mathsf{T}} } + \frac{\partial \mathcal{l} \left( \mathbf{\theta} \right) }{ \partial \mathbf{\theta} } \frac{\partial \mathcal{l} \left( \mathbf{\theta} \right) }{ \partial \mathbf{\theta} } \right]</math>

:<math>\mathbf{\Delta}(\mathbf{\theta}) = \sum_{i=1}^n \left[ \frac{\partial^2 \ell(\mathbf{\theta}) }{ \partial \mathbf{\theta} \, \partial \mathbf{\theta}^{\mathsf{T}} } + \frac{\partial \ell(\mathbf{\theta}) }{ \partial \mathbf{\theta} } \frac{\partial \ell (\mathbf{\theta}) }{ \partial \mathbf{\theta} } \right]</math>

where <math>\mathbf{\Delta} \left( \mathbf{\theta} \right)</math> is an <math>(r \times r)</math> [[random matrix]], where <math>r</math> is the number of parameters. White showed that the elements of <math>n^{-\frac{1}{/2}} \mathbf{\Delta} ( \mathbf{\hat{\theta}} )</math>, where <math>\mathbf{\hat{\theta}}</math> is the MLE, are asymptotically [[Normal distribution|normally distributed]] with zero means when the model is correctly specified.<ref>{{cite book |first=L. G. |last=Godfrey |title=Misspecification Tests in Econometrics |___location=New York |publisher=Cambridge University Press |year=1988 |isbn=0-521-26616-5 |pages=35–37 }}</ref>

== References ==