Generalized linear model

In statistics the generalized linear model (GLM) generalizes the ordinary least squares regression. A GLM can be written as follows:

The data (${\displaystyle \mathbf {Y} }$) are predicted by

${\displaystyle E(\mathbf {Y} )={\boldsymbol {\mu }}=g^{-1}(X\beta )}$

where ${\displaystyle g}$ is called the link function and the random component is a function of the mean

${\displaystyle Var(Y)=f({\boldsymbol {\mu }})}$

It is convenient if the variance follows from the exponential family which covers a very large range of distributions, but it may also simply be a statement that the variance is a function of the predicted value.

Parameters must be estimated with maximum likelihood or quasi maximum likelihood.

Examples

The simplest example of a GLM is linear regression. Here the link function is the identity and the variance is assumed to be normally distributed.

Binomial data

When the response data (${\displaystyle Y}$ ) is binary, the variance is generally regarded as binomial and the interpretation of ${\displaystyle \mu _{i}}$  is then the probability of ${\displaystyle Y_{i}}$  taking on the value one. The variance function is given by:

${\displaystyle Var(Y)=\phi \mu (1-\mu )}$ 

where ${\displaystyle \phi }$  is often exactly one. When it is not, the variance is often described as quasibinomial.

There are many popular link functions for binomial functions, they include the logistic function:

${\displaystyle g(p)=\log \left({p \over 1-p}\right)}$ 

In addition, any cumulative density function can be used and the normal is a popular choice and is called the probit model, and the link is

${\displaystyle g(p)=\Phi ^{-1}(p)}$ 

where ${\displaystyle \Phi }$  is the cumulative density function of the normal distribution

The identity link is also sometimes used for binomial data (this is equivalent to using the uniform distribution instead of the normal as the CDF) but this encounters problems when the predicted probabilities are greater or less than one. In implementation this is possible to fix but interpreting the coefficients can be difficult in this model. But, it is not too distant from the logit or probit around p=0.5.

Count data

Another example of generalized linear models includes Poisson regression which models count data. In this case, the variance is proportional to the mean

${\displaystyle Var(y)=\phi \mu }$ 

where ${\displaystyle \phi }$ , the dispersion parameter, is often equal to one. When it is not, the variance is often described as poisson with overdispersion or quasipoisson.

References

• P. McCullagh and J.A. Nelder. Generalized Linear Models. London: Chapman and Hall, 1989.

• A.J. Dobson. Introduction to Generalized Linear Models, Second Edition. London: Chapman and Hall/CRC, 2001.

External links

• John Nelder FRS
• Royal Society citation for Nelder