Reproducing kernel Hilbert space: Difference between revisions

Revision as of 02:44, 24 January 2023

Revision as of 06:34, 31 January 2023

Line 1:

[[File:Different Views on RKHS.png|thumb|right|Figure illustrates related but varying approaches to viewing RKHS]]

In [[functional analysis]] (a branch of [[mathematics]]), a '''reproducing kernel Hilbert space''' ('''RKHS''') is a [[Hilbert space]] of functions in which point evaluation is a continuous linear [[Functional (mathematics)|functional]]. Roughly speaking, this means that if two functions <math>f</math> and <math>g</math> in the RKHS are close in norm, i.e., <math>\|f-g\|</math> is small, then <math>f</math> and <math>g</math> are also pointwise close, i.e., <math>|f(x)-g(x)|</math> is small for all <math>x</math>. The converse does not need to be true. Informally, this can be shown by looking at the [[Uniform norm|supremum norm]]: the sequence of functions <math>\sin^n (x)</math> converges pointwise, but do not converge [[Uniform Convergence|uniformly]] i.e. do not converge with respect to the supremum norm (note that this is not a counterexample because the supremum norm does not arise from any [[inner product|inner product]] due to not satisfying the [[Polarization identity|parallelogram law]]).

It is not entirely straightforward to construct a Hilbert space of functions which is not an RKHS.<ref>Alpay, D., and T. M. Mills. "A family of Hilbert spaces which are not reproducing kernel Hilbert spaces." J. Anal. Appl. 1.2 (2003): 107–111.</ref> Some examples, however, have been found.<ref> Z. Pasternak-Winiarski, "On weights which admit reproducing kernel of Bergman type", ''International Journal of Mathematics and Mathematical Sciences'', vol. 15, Issue 1, 1992. </ref><ref> T. Ł. Żynda, "On weights which admit reproducing kernel of Szeg¨o type", ''Journal of Contemporary Mathematical Analysis'' (Armenian Academy of Sciences), 55, 2020. </ref>

Note that [[Square-integrable function|''L''² spaces]] are not Hilbert spaces of functions (and hence not RKHSs), but rather Hilbert spaces of equivalence classes of functions (for example, the functions <math>f</math> and <math>g</math> defined by <math>f(x)=0</math> and <math>g(x)=1_{\mathbb{Q}}</math> are equivalent in ''L''²). However, there are RKHSs in which the norm is an ''L''²-norm, such as the space of band-limited functions (see the example below).

An RKHS is associated with a kernel that reproduces every function in the space in the sense that for every <math>x</math> in the set on which the functions are defined, "evaluation at <math>x</math>" can be performed by taking an inner product with a function determined by the kernel. Such a ''reproducing kernel'' exists if and only if every evaluation functional is continuous.

Line 73:

:<math>K_x(y) = \frac{a}{\pi} \operatorname{sinc}\left ( \frac{a}{\pi} (y-x) \right )=\frac{\sin(a(y-x))}{\pi(y-x)}.</math>

To see this, we first note that theThe Fourier transform of <math>K_x(y)</math> defined above is given by

:<math>\int_{-\infty}^\infty K_x(y)e^{-i \omega y} \, dy =

Line 89:

Thus we obtain the reproducing property of the kernel.

Note that <math>K_x</math> in this case is the "bandlimited version" of the [[Dirac delta function]], and that <math>K_x(y)</math> converges to <math>\delta(y-x)</math> in the weak sense as the cutoff frequency <math>a</math> tends to infinity.

== Moore–Aronszajn theorem ==

Line 157:

A '''feature map''' is a map <math> \varphi\colon X \rightarrow F </math>, where <math> F </math> is a Hilbert space which we will call the feature space. The first sections presented the connection between bounded/continuous evaluation functions, positive definite functions, and integral operators and in this section we provide another representation of the RKHS in terms of feature maps.

We first note that everyEvery feature map defines a kernel via

{{NumBlk|:|<math> K(x,y) = \langle \varphi(x), \varphi(y) \rangle_F. </math> |{{EquationRef|3}}}}

Line 237:

:<math> \langle f, \Gamma_x c \rangle_H = f(x)^\intercal c. </math>

This second property parallels the reproducing property for the scalar-valued case. We note that thisThis definition can also be connected to integral operators, bounded evaluation functions, and feature maps as we saw for the scalar-valued RKHS. We can equivalently define the vvRKHS as a vector-valued Hilbert space with a bounded evaluation functional and show that this implies the existence of a unique reproducing kernel by the Riesz Representation theorem. Mercer's theorem can also be extended to address the vector-valued setting and we can therefore obtain a feature map view of the vvRKHS. Lastly, it can also be shown that the closure of the span of <math> \{ \Gamma_xc : x \in X, c \in \mathbb{R}^T \} </math> coincides with <math> H </math>, another property similar to the scalar-valued case.

We can gain intuition for the vvRKHS by taking a component-wise perspective on these spaces. In particular, we find that every vvRKHS is isometrically [[isomorphic]] to a scalar-valued RKHS on a particular input space. Let <math>\Lambda = \{1, \dots, T \} </math>. Consider the space <math> X \times \Lambda </math> and the corresponding reproducing kernel