Multiple kernel learning: Difference between revisions

Revision as of 23:19, 17 December 2014

Revision as of 23:20, 17 December 2014

Line 11:

Multiple kernel learning algorithms have been developed for supervised, semi-supervised, as well as unsupervised learning. Most work has been done on the supervised learning case with linear combinations of kernels, however, many algorithms have been developed. The basic idea behind multiple kernel learning algorithms is to add an extra parameter to the minimization problem of the learning algorithm. As an example, consider the case of supervised learning of a linear combination of a set of <math>n</math> kernels <math>K</math>. We introduce a new kernel <math>K'=\sum_{i=1}^n\beta_iK_i</math>, where <math>\beta_i</math> is a vector of coefficients for each kernel. Because the kernels are additive (due to properties of [[reproducing kernel Hilbert spaces]]), this new function is still a kernel. For a set of data <math>X</math> with labels <math>Y</math>, the minimization problem can then be written as

:<math>\min_{\beta,c}\Epsilon(Y, K'c)+R(K,c)</math>

where <math>\Epsilon</math> is an error function and <math>R</math> is a regularization term. <math>\Epsilon</math> is typically the square loss function ([[Tikhonov regularization]]) or the hinge loss function (for [[Support vector machine|SVM]] algorithms), and <math>R</math> is usually an <math>\ell_n</math> norm or some combination of the norms (i.e. [[elastic net regularization]]). This optimization problem can then be solved by standard optimization methods. Adaptations of existing techniques such as the Sequential Minimal Optimization have also been developed for multiple kernel SVM-based methods <ref name = "conic">Francis R. Bach, Gert R. G. Lanckriet, and Michael I. Jordan. 2004. [http://machinelearning.wustl.edu/mlpapers/paper_files/icml2004_BachLJ04.pdf Multiple kernel learning, conic duality, and the SMO algorithm]. In Proceedings of the twenty-first international conference on Machine learning (ICML '04). ACM, New York, NY, USA</ref>.