Variance function

Variance Function new article content ...

Not to be confused with Variance.

In statistics, the variance function is a function relating the variance of a random quantity to the conditional mean of the random quantity. The variance function is a main ingredient in the generalized linear model framework and plays roles in Non-parametric regression and Functional data analysis as well. Not to be confused with the variance of a function, in parametric modelling, variance functions explicitly describe the relationship between the variance and the conditional mean of a random variable. For many well known distributions, the variance function represents the complete variance of a random variable under that distribution, but in fact, these are just special cases.

Intuition

Overview

Types

The variance function and it's applications comes up in many areas of statistical analysis. A very important use of this function is in the framework of Generalized linear models and Nonparametric regression.

Generalized Linear Model

Derivation

The Generalized linear model, GLM, is a generalization of ordinary regression analysis that extends to any member of the exponential family. It is particularly useful when the response variable is categorical, binary or subject to a constraint (e.g. only positive responses make sense). A quick summary of the components of a GLM are summarized on this page, but for more details and information see the page on generalized linear models.

Any random variable ${\displaystyle {\textbf {y}}}$  in the exponential family has a probability density function of the form,

${\displaystyle \operatorname {f} (y,\theta ,\phi )=exp\left({\frac {y\theta -b(\theta )}{\phi }}-c(y,\phi )\right)}$ 

with loglikelihood,

${\displaystyle \operatorname {l} (\theta ,y,\phi )=\log(\operatorname {f} (y,\theta ,\phi )={\frac {y\theta -b(\theta )}{\phi }}-c(y,\phi )}$ 

Here, ${\displaystyle \theta }$  is the canonical parameter and the parameter of interest, and ${\displaystyle \phi }$  is a nuisance parameter which plays a role in the variance. We use the Bartlett's Identities insert reference to derive a general expression for the variance function. The first and second Bartlett results ensures that under suitable conditions ( insert references), for a density function dependent on &theta, ${\displaystyle f_{\theta }(.)}$ ,

${\displaystyle \operatorname {E} _{\theta }\left[{\frac {\partial }{\partial \theta }}\log(f_{\theta }(y))\right]=0}$ 

${\displaystyle \operatorname {Var} _{\theta }\left[{\frac {\partial }{\partial \theta }}\log(f_{\theta }(y))\right]+\operatorname {E} _{\theta }\left[{\frac {\partial ^{2}}{\partial \theta ^{2}}}\log(f_{\theta }(y))\right]=0}$ 

These identities lead to simple calculations of the expected value and variance of any random variable ${\displaystyle {\textbf {y}}}$  in the exponential family ${\displaystyle E_{\theta }[y],Var_{\theta }[y]}$ . Expected Value of y Taking the first derivative with respect to ${\displaystyle \theta }$  of the log of the density in the exponential family form described above, we have

${\displaystyle {\frac {\partial }{\partial \theta }}\log(\operatorname {f} (y,\theta ,\phi )={\frac {\partial }{\partial \theta }}\left[{\frac {y\theta -b(\theta )}{\phi }}-c(y,\phi )\right]={\frac {y-b'(\theta )}{\phi }}}$ 

Then taking the expected value and setting it equal to zero leads to,

${\displaystyle \operatorname {E} _{\theta }\left[{\frac {y-b'(\theta )}{\phi }}\right]={\frac {\operatorname {E} _{\theta }[y]-b'(\theta )}{\phi }}=0}$ 

${\displaystyle \operatorname {E} _{\theta }[y]=b'(\theta )}$ 

Variance of y To compute the variance we use the second Bartlett identity,

${\displaystyle \operatorname {Var} _{\theta }\left[{\frac {\partial }{\partial \theta }}\left({\frac {y\theta -b(\theta )}{\phi }}-c(y,\phi )\right)\right]+\operatorname {E} _{\theta }\left[{\frac {\partial ^{2}}{\partial \theta ^{2}}}\left({\frac {y\theta -b(\theta )}{\phi }}-c(y,\phi )\right)\right]=0}$ 

${\displaystyle \operatorname {Var} _{\theta }\left[{\frac {y-b'(\theta )}{\phi }}\right]+\operatorname {E} _{\theta }\left[{\frac {-b''(\theta )}{\phi }}\right]=0}$ 

${\displaystyle \operatorname {Var} _{\theta }\left[y\right]=b''(\theta )\phi }$ 

</math>

Examples

Normal

Binomial

Poisson

Gamma

Application

Maximum Likelihood Estimation

Quasi Likelihood

Non-Parametric Regression Analysis

See Also

References

External links

• example.com

Generalized linear models Quasi Likelihood Non-Parametric Regression