Reproducing kernel Hilbert space: Difference between revisions

Revision as of 02:28, 13 August 2007

Revision as of 11:15, 13 August 2007

Line 31:

== Moore-Aronszajn theorem ==

Given a [[positive definite kernel]] <math>''K</math>'', we can construct a unique RKHS <math>''H</math>'' with <math>''K</math>'' as the reproducing kernel. A positive definite kernel is a function on <math>X \otimes X</math> with the following property.

For all natural number <math>n</math>, for all <math>x_1, \ldots, x_n</math> in <math>X</math>, and for all <math>\alpha_1, \ldots, \alpha_n</math> in a real or complex,

Define the operator <math>T_K : L_2 (X) \rightarrow L_2 (X)</math> by

:<math> \sum_{i=1}^n \sum_{j=1}^n \alpha_i \alpha_j K(x_i, x_j) \ge 0.</math>

:<math> (T_K f) (x) = \int_X \overline {K (x, y)} f(y)\,dy </math>

or, equivalently, <math> (T_K f)(x) = \langle K_x, f \rangle_2 </math>

(where <math>\langle \cdot, \cdot \rangle_2 </math> denotes the standard inner product on ''L''₂''(X)'').

Let ''H'' be the image of ''L''₂ ''(X)'' under ''T''_''K'' and define an inner product on ''H'' by

:<math> \langle f, g \rangle_H \ \stackrel{\mathrm{def}}{=} \ \langle f, T_K^{-1} g \rangle_2. </math>

Note that ''T''_''K''^-1 is self-adjoint (we will write down exactly what it is later) and so

<math> \langle f, T_K^{-1} g \rangle_2 = \langle T_K^{-1} f, g \rangle_2 </math>. It is now easy to check that this defines a reproducing kernel Hilbert space. Indeed,

:<math>

\langle K_x, f \rangle_H = \langle K_x, T_K^{-1} f \rangle_2

= (T_K (T_K^{-1} f)) (x) = f(x)

</math>

as required.

[[Mercer's theorem]] gives us another way to represent ''H''. Let {&lambda;_''i''} be a sequence of eigenvalues of ''T''_''K'' and let {''e''_''i''} be the corresponding eigenvectors. Then we can write the operator ''T''_''K'' as

:<math> T_K \left( \sum_i a_i e_i \right) = \sum_i \lambda_i a_i e_i

</math>

and we can write the inner product on ''H'' as

:<math> \left \langle \sum_i a_i e_i, \sum_i b_i e_i \right \rangle_H = \sum_i \frac {a_i b_i}{\lambda_i} </math>.

== See also ==