Multivariate kernel density estimation: Difference between revisions

Revision as of 16:44, 15 September 2010

Revision as of 20:57, 15 September 2010

Line 1:

{{Userspace draft|source=ArticleWizard|date=September 2010}}

[[Kernel density estimation]] is one of the most popular techniques for [[density estimation]] i.e., estimation of [[probability density function|probability density functions]] which is one of the fundamental questions in [[statistics]].

[[Kernel density estimation]] is one of the most popular techniques for density estimation. It can be viewed as a generalisation of [[histogram]] density estimation with improved statistical properties.

[[Statistics]] is a field of quantitative analysis concerned with quantifying uncertainty. The main building block of statistical analysis is a [[random variable]]. A random variable is a [[mathematical|mathematics]] function which assigns a numerical value to each possible value of

Kernel density estimators were first introduced in the scientific literature for [[univariate]] data in the 1950s and 1960s<ref>{{cite journal | doi=10.1214/aoms/1177728190 | last=Rosenblatt | first=M.| title=Remarks on some nonparametric estimates of a density function | journal=Annals of Mathematical Statistics | year=1956 | volume=27 | pages=832-837}}</ref><ref>{{cite journal | doi=10.1214/aoms/1177704472| last=Parzen | first=E.| title=On estimation of a probability density function and mode | journal=Annals of Mathematical Statistics| year=1962 | volume=33 | pages=1065-1076}}</ref> and subsequently have been widely adopted. It was soon recognised that analagousanalogous estimators for multivariate data would be an important addition to [[multivariate statistics]]. Based on research carried out in the 1990s and 2000s, multivariate kernel density estimation has reached a level of maturity comparable to their univariate counterparts.<ref>{{cite book | author=Simonoff, J.S. | title=Smoothing Methods in Statistics | publisher=Springer | date=1996 | isbn=0387947167}}</ref>

the variable of interest. The complete behaviour of a random variable is contained in its [[distribution function]]. For [[continuous]] random variables, the partial derivative of the distribution function is known as [[probability density function]] or pdf. So [[density estimation]] is a fundamental question in statistics.

[[Kernel density estimation]] is one of the most popular techniques for density estimation. It can be viewed as a generalisation of [[histogram]] density estimation with improved statistical properties.

Kernel density estimators were first introduced in the scientific literature for [[univariate]] data in the 1950s and 1960s<ref>{{cite journal | doi=10.1214/aoms/1177728190 | last=Rosenblatt | first=M.| title=Remarks on some nonparametric estimates of a density function | journal=Annals of Mathematical Statistics | year=1956 | volume=27 | pages=832-837}}</ref><ref>{{cite journal | doi=10.1214/aoms/1177704472| last=Parzen | first=E.| title=On estimation of a probability density function and mode | journal=Annals of Mathematical Statistics| year=1962 | volume=33 | pages=1065-1076}}</ref> and subsequently have been widely adopted. It was soon recognised that analagous estimators for multivariate data would be an important addition to [[multivariate statistics]]. Based on research carried out in the 1990s and 2000s, multivariate kernel density estimation has reached a level of maturity comparable to their univariate counterparts.

Line 18 ⟶ 15:

Let <math>\bold{X}_1, \bold{X}_2, \dots, \bold{X}_n</math> be a <em>d</em>-variate random sample drawn from a common density function <em>f</em>. The kernel density estimate is defined to be

<math>\hat{f}_\bold{H}(\bold{x})= n^{-1} |\bold{H}|^{-1/2} \sum_{i=1}^n K_\bold{H} (\bold{x} - \bold{X}_i)</math>

where

Line 39 ⟶ 36:

<math>\operatorname{AMISE} (\bold{H}) = n^{-1} |\bold{H}|^{-1/2} R(K) + \tfrac{1}{4} m_2(K)^2

\int (\operatorname{trvec}^2T (\bold{H}) \operatornamebold{D\Psi}^2_4 f(\boldoperatorname{xvec})) \,^T d\bold{xH})</math>

where

<ul>

<li><math>R(K) = \int K(\bold{x})^2 \, d\bold{x}</math>., For the normal kernel <math>K</math>,with <math>R(K) = (4 \pi)^{-d/2}</math> when <math>K</math> is a normal kernel

<li><math>\int \bold{x} \bold{x}^T K(\bold{x})^2 \, d\bold{x}</math> = m_2(K) \bold{I}_d</math>,

with <mathstrong>\bold{I}_d</mathstrong>_d isbeing the <em>d x d</em> [[identity matrix]]

<li><math>\operatorname{D}^2 f</math> is the <em>d x d</em> Hessian matrix of second order partial derivatives of <math>f</math>

<li><math>\bold{\Psi}_4 = \int (\operatorname{vec} \, \operatorname{D}^2 f(\bold{x})) (\operatorname{vec}^T \operatorname{D}^2 f(\bold{x})) \, d\bold{x}</math>

<li>vec is the vector operator which stacks the columns of a matrix into a single vector e.g. <math>\operatorname{vec}\begin{bmatrix}a & c \\ b & d\end{bmatrix} = \begin{bmatrix}a & b & c & d\end{bmatrix}^T.</math>

</ul>

This formula of the AMISE is due to <ref name="WJ1995">/</ref>(p. 97). The quality of the AMISE approximation to the MISE is given by

Line 57 ⟶ 56:

<math>\bold{H}_{\operatorname{AMISE}} = \operatorname{argmin}_{\bold{H} \in F} \, \operatorname{AMISE} (\bold{H})</math>

where <em>F</em> is the space of all symmetric, positive definite matrices.

Since this ideal selector contains the unknown density function ><em>f</em>, it cannot be used directly. The many different varieties of data-based bandwidth selectors arise from the different estimators of the AMISE. We concentrate on two classes of selectors which have been shown to be the most widely applicable in practise: smoothed cross validation and plug-in selectors.

=== Plug-in ===

Line 82 ⟶ 81:

waiting time until the next eruption (minutes), and is contained in the base distribution of R.

This code snippetfragment computes the kernel density estimate with the plug-in bandwidth matrix. The coloured contours correspond to the smallest region which contains that corresponding probability mass: red = 25%, orange + red = 50%, yellow + orange + red = 75%. To compute the SCV selector, replace <code>Hpi</code> is replaced with <code>Hscv</code>. This is not displayed here since it is mostly similar to the plug-in estimate for this data set.

<pre>

Line 100 ⟶ 99:

== External links ==

* [http://www.mvstat.net/tduong/research www.mvstat.net/tduong/research] WebA pagecollection containingof peer-reviewed articles of the mathematical details of multivariate kernel density estimation and their bandwidth selectors.

==See also==

*[[Kernel density estimation]] Univariate kernel density estimation.

*[[Kernel smoothing]] Univariate kernel regression.