Multivariate kernel density estimation: Difference between revisions

Revision as of 02:10, 24 December 2014

Revision as of 21:46, 12 February 2015

Line 52:

where ''o'' indicates the usual [[big O notation|small o notation]]. Heuristically this statement implies that the AMISE is a 'good' approximation of the MISE as the sample size <em>n → ∞<em>.

It can be shown that any reasonable bandwidth selector '''H''' has '''H''' = ''O(n^-2−2/(d+4))'' where the [[big O notation]] is applied elementwise. Substituting this into the MISE formula yields that the optimal MISE is ''O(n^-4−4/(d+4)).''<ref name="WJ1995"/>{{rp|99–100}} Thus as ''n → ∞'', the MISE → 0, i.e. the kernel density estimate [[convergence in mean|converges in mean square]] and thus also in probability to the true density ''f''. These modes of convergence are confirmation of the statement in the motivation section that kernel methods lead to reasonable density estimators. An ideal optimal bandwidth selector is

: <math>\bold{H}_{\operatorname{AMISE}} = \operatorname{argmin}_{\bold{H} \in F} \, \operatorname{AMISE} (\bold{H}).</math>

Line 90:

:<math>\operatorname{Var} \hat{f}(\bold{x};\bold{H}) = n^{-1} |\bold{H}|^{-1/2} R(K) + o(n^{-1} |\bold{H}|^{-1/2}).</math>

For these two expressions to be well-defined, we require that all elements of '''H''' tend to 0 and that ''n^-1−1'' |'''H'''|^-1−1/2 tends to 0 as ''n'' tends to infinity. Assuming these two conditions, we see that the expected value tends to the true density ''f'' i.e. the kernel density estimator is asymptotically [[Bias of an estimator|unbiased]]; and that the variance tends to zero. Using the standard mean squared value decomposition

:<math>\operatorname{MSE} \, \hat{f}(\bold{x};\bold{H}) = \operatorname{Var} \hat{f}(\bold{x};\bold{H}) + [\operatorname{E} \hat{f}(\bold{x};\bold{H}) - f(\bold{x})]^2</math>

we have that the MSE tends to 0, implying that the kernel density estimator is (mean square) consistent and hence converges in probability to the true density ''f''. The rate of convergence of the MSE to 0 is the necessarily the same as the MISE rate noted previously ''O(n^-4−4/(d+4))'', hence the covergence rate of the density estimator to ''f'' is ''O_p(n^-2−2/(d+4))'' where ''O_p'' denotes [[Big O in probability notation|order in probability]]. This establishes pointwise convergence. The functional covergence is established similarly by considering the behaviour of the MISE, and noting that under sufficient regularity, integration does not affect the convergence rates.

For the data-based bandwidth selectors considered, the target is the AMISE bandwidth matrix. We say that a data-based selector converges to the AMISE selector at relative rate ''O_p(n^-α−α), α > 0'' if

:<math>\operatorname{vec} (\hat{\bold{H}} - \bold{H}_{\operatorname{AMISE}}) = O(n^{-2\alpha}) \operatorname{vec} \bold{H}_{\operatorname{AMISE}}.</math>

It has been established that the plug-in and smoothed cross validation selectors (given a single pilot bandwidth '''G''') both converge at a relative rate of ''O_p(n^-2−2/(d+6))'' <ref name="DH2005"/><ref>{{Cite journal| doi=10.1016/j.jmva.2004.04.004 | author1=Duong, T. | author2=Hazelton, M.L. | title=Convergence rates for unconstrained bandwidth matrix selectors in multivariate kernel density estimation | journal=Journal of Multivariate Analysis | year=2005 | volume=93 | pages=417–433}}</ref> i.e., both these data-based selectors are consistent estimators.

==Density estimation in R with a full bandwidth matrix==