Reproducing kernel Hilbert space

In functional analysis (a branch of mathematics), a reproducing kernel Hilbert space is a function space in which pointwise evaluation is a continuous linear functional. Equivalently, they are spaces that can be defined by reproducing kernels. The subject was originally and simultaneously developed by Nachman Aronszajn (1907-1980) and Stephan Bergman (1895-1987) in 1950.

In this article we assume that Hilbert spaces are complex. This is because many of the examples of reproducing kernel Hilbert spaces are spaces of analytic functions. Also recall the sesquilinearity convention: the inner product is linear in the second variable.

Let X be an arbitrary set and H a Hilbert space of complex-valued functions on X. H is a reproducing kernel Hilbert space iff the linear map

${\displaystyle f\mapsto f(x)}$

from H to the complex numbers is continuous for any x in X. By the Riesz representation theorem, this implies that for given x there exists an element K_x of H with the property that:

${\displaystyle f(x)=\langle K_{x},f\rangle \quad \forall f\in H\quad (*)}$

The function

${\displaystyle K(x,y)\ {\stackrel {\mathrm {def} }{=}}\ K_{x}(y)}$

is called a reproducing kernel for the Hilbert space. In fact, K is uniquely determined by the above condition (*).

For example, when X is finite and H consists of all complex-valued functions on X, then an element of H can be represented as an array of complex numbers. If the usual inner product is used, then K_x is the function whose value is 1 at x and 0 everywhere else.

In other contexts, (*) amounts to saying

${\displaystyle f(x)=\int _{X}K(x,y)f(y)\,dy}$

for every f, where X is often the real numbers or Rⁿ.

Bergman kernel

The Bergman kernel is defined for open sets D in Cⁿ. Take the Hilbert H space of square-integrable functions, for the Lebesgue measure on D, that are holomorphic functions. The theory is non-trivial in such cases as there are such functions, which are not identically zero. Then H is a reproducing kernel space, with kernel function the Bergman kernel; this example, with n = 1, was introduced by Bergman in 1922.

Moore-Aronszajn theorem

Given a positive definite kernel K, we can construct a unique RKHS H with K as the reproducing kernel.

Define the operator ${\displaystyle T_{K}:L_{2}(X)\rightarrow L_{2}(X)}$  by

${\displaystyle (T_{K}f)(x)=\int _{X}{\overline {K(x,y)}}f(y)\,dy}$ 

or, equivalently, ${\displaystyle (T_{K}f)(x)=\langle K_{x},f\rangle _{2}}$  (where ${\displaystyle \langle \cdot ,\cdot \rangle _{2}}$  denotes the standard inner product on L₂(X)). Let H be the image of L₂ (X) under T_K and define an inner product on H by

${\displaystyle \langle f,g\rangle _{H}\ {\stackrel {\mathrm {def} }{=}}\ \langle f,T_{K}^{-1}g\rangle _{2}.}$ 

Note that T_K^-1 is self-adjoint (we will write down exactly what it is later) and so ${\displaystyle \langle f,T_{K}^{-1}g\rangle _{2}=\langle T_{K}^{-1}f,g\rangle _{2}}$ . It is now easy to check that this defines a reproducing kernel Hilbert space. Indeed,

${\displaystyle \langle K_{x},f\rangle _{H}=\langle K_{x},T_{K}^{-1}f\rangle _{2}=(T_{K}(T_{K}^{-1}f))(x)=f(x)}$ 

as required.

Mercer's theorem gives us another way to represent H. Let {λ_i} be a sequence of eigenvalues of T_K and let {e_i} be the corresponding eigenvectors. Then we can write the operator T_K as

${\displaystyle T_{K}\left(\sum _{i}a_{i}e_{i}\right)=\sum _{i}\lambda _{i}a_{i}e_{i}}$ 

and we can write the inner product on H as

${\displaystyle \left\langle \sum _{i}a_{i}e_{i},\sum _{i}b_{i}e_{i}\right\rangle _{H}=\sum _{i}{\frac {a_{i}b_{i}}{\lambda _{i}}}}$ .

See also

• Positive definite kernel
• Mercer's theorem
• Kernel trick

References

• Nachman Aronszajn, Theory of Reproducing Kernels, Transactions of the American Mathematical Society, volume 68, number 3, pages 337-404, 1950.