Reproducing kernel Hilbert space: Difference between revisions

Revision as of 14:15, 17 August 2007

Revision as of 08:14, 20 August 2007

Line 31:

== Moore-Aronszajn theorem ==

The following theorem is mentioned by Aronszajn in ''Theory of Reproducing Kernels'', although he attributes it to [[E. H. Moore]].

Given a [[positive definite kernel]] <math>K</math>, we can construct a unique RKHS <math>H</math> with <math>K</math> as the reproducing kernel. A positive definite kernel is a function on <math>X \otimes X</math> with the following property.

For all natural number <math>n</math>, for all <math>x_1, \ldots, x_n</math> in <math>X</math>, and for all <math>\alpha_1, \ldots, \alpha_n</math> in a real or complex,

'''Theorem'''.

:<math> \sum_{i=1}^n \sum_{j=1}^n \alpha_i \alpha_j K(x_i, x_j) \ge 0.</math>

Suppose ''K'' is a [[positive definite kernel]] on a set ''E''. Then there is a unique Hilbert space of functions on ''E'' for which ''K'' is a reproducing kernel.

'''Proof'''.

Define, for all ''x'' in ''E'', <math>K_x = K(x, \cdot)</math>.

Let ''H''₀ be the linear span of <math> \{K_x:\ x \in E \} </math>.

Define an inner product on ''H''₀ by

:<math>

\left \langle \sum_{i=1}^m a_i K_{x_i}, \sum_{j=1}^n b_j K_{y_j} \right \rangle = \sum_{i=1}^m \sum_{j=1}^n a_i b_j K(y_j, x_i).

</math>

Let ''H'' be the completion of ''H''₀ with respect to this inner product. Then ''H'' consists of functions of the form

:<math>

f(x) = \sum_{i=1}^\infty a_i K_{x_i} (x)

</math>

where <math>\sum_{i=1}^\infty a_i^2 K (x_i, x_i) < \infty</math>. The fact that the above sum converges for every ''x'' follows from the Cauchy-Schwartz inequality.

Now we can check the RKHS property, (*):

:<math>

\langle K_x, f \rangle = \left \langle K_x, \sum_{i=1}^\infty a_i K_{x_i} \right \rangle

= \sum_{i=1}^\infty a_i K (x_i, x) = f(x).

</math>

To prove uniqueness, let ''G'' be another Hilbert space of functions for which ''K'' is a reproducing kernel. For any ''x'' and ''y'' in ''E'',

(*) implies that

<math>\langle K_x, K_y \rangle_H = K(x, t) = \langle K_x, K_y \rangle_G</math>.

By linearity, <math>\langle \cdot, \cdot \rangle_H = \langle \cdot, \cdot \rangle_G</math> on the span of <math> \{K_x:\ x \in E \} </math>. Then ''G'' = ''H'' by the uniqueness of the completion.

== See also ==

Line 45 ⟶ 70:

* Nachman Aronszajn, ''Theory of Reproducing Kernels'', Transactions of the American Mathematical Society, volume 68, number 3, pages 337-404, 1950.

* Felipe Cucker and Steve Smale, ''On the Mathematical Foundations of Learning'', Bulletin of the American Mathematical Society, volume 39, number 1, pages 1-49, 2001.

[[Category:Hilbert space]]