Kernel (machine learning)

In the field of machine learning , a class of algorithms has been developed that use a kernel to implicitly carry out their calculations in a higher-dimensional space. Well-known algorithms that work with kernels are the support vector machines and the kernel PCA .

In this context, one speaks of the kernel trick , because with this method a linear classifier is applied to data that cannot be classified linearly. This is achieved by transforming the data into a higher-dimensional space in which one hopes for better linear separability.

Formal definition

Be an input space. An illustration is the kernel when there is an inner product and an image are in this room with: . is called feature space or feature space, feature mapping or feature mapping. In practice, the feature space does not have to be known explicitly, since kernels have a simple characterization through Mercer's theorem . ${\displaystyle X}$ ${\displaystyle K\colon X\times X\to \mathbb {R} }$ ${\displaystyle (F,\langle \cdot ,\cdot \rangle )}$ ${\displaystyle \phi \colon X\to F}$ ${\displaystyle K(x,y)=\langle \phi (x),\phi (y)\rangle \;\;\;\;\forall x,y\in X}$ ${\displaystyle F}$ ${\displaystyle \phi }$

Different classes of kernel functions

There are different types of kernels, some of which can be adapted to the given problem using parameters:

• linear kernel ${\displaystyle k(x,y)=\langle x,y\rangle }$
• polynomial kernels ${\displaystyle k(x,y)=\langle x,y\rangle ^{d}}$
• RBF kernel ${\displaystyle k(x,y)=\exp \left(-{\tfrac {||x-y||^{2}}{2\sigma ^{2}}}\right)}$

In the meantime, kernels have also been defined on graphs and strings.

literature

• Christopher M. Bishop: Pattern Recognition and Machine Learning . Information Science and Statistics, Springer-Verlag, 2008, ISBN 978-0387310732
• Nello Cristianini, John Shawe-Taylor: Kernel Methods for Pattern Classification . Cambridge, 2004.
• Bernhard Schölkopf , Alex Smola: Learning with Kernels . MIT Press, Cambridge, MA, 2002.
• Thomas Hofmann, Bernhard Schölkopf, Alexander J Smola: Kernel methods in machine learning. In: Annals Statistics 36 (3) 2008: 1171-1220. PDF.

Web links

• http://www.kernel-machines.org/
• http://www.kernel-methods.net