Kernels

Kernels

Mercer Kernels

MLKernels.ExponentialKernel — Type.
ExponentialKernel([α=1])

The exponential kernel is given by the formula:

\[\kappa(\mathbf{x},\mathbf{y}) = \exp\left(-\alpha ||\mathbf{x} - \mathbf{y}||\right) \qquad \alpha > 0\]

where $\alpha$ is a scaling parameter of the Euclidean distance. The exponential kernel, also known as the Laplacian kernel, is an isotropic Mercer kernel. The constructor is aliased by LaplacianKernel, so both names may be used:

source
MLKernels.SquaredExponentialKernel — Type.
SquaredExponentialKernel([α=1])

The squared exponential kernel, or alternatively the Gaussian kernel, is identical to the exponential kernel except that the Euclidean distance is squared:

\[\kappa(\mathbf{x},\mathbf{y}) = \exp\left(-\alpha ||\mathbf{x} - \mathbf{y}||^2\right) \qquad \alpha > 0\]

where $\alpha$ is a scaling parameter of the squared Euclidean distance. Just like the exponential kernel, the squared exponential kernel is an isotropic Mercer kernel. The squared exponential kernel is more commonly known as the radial basis kernel within machine learning communities.

source
MLKernels.GammaExponentialKernel — Type.
GammaExponentialKernel([α=1 [,γ=1]])

The gamma exponential kernel is a generalization of the exponential and squared exponential kernels:

\[\kappa(\mathbf{x},\mathbf{y}) = \exp\left(-\alpha ||\mathbf{x} - \mathbf{y}||^{\gamma} \right) \qquad \alpha > 0, \; 0 < \gamma \leq 1\]

where $\alpha$ is a scaling parameter and $\gamma$ is a shape parameter.

source
MLKernels.RationalQuadraticKernel — Type.
RationalQuadraticKernel([α=1 [,β=1]])

The rational-quadratic kernel is given by:

\[\kappa(\mathbf{x},\mathbf{y}) = \left(1 +\alpha ||\mathbf{x},\mathbf{y}||^2\right)^{-\beta} \qquad \alpha > 0, \; \beta > 0\]

where $\alpha$ is a scaling parameter and $\beta$ is a shape parameter. This kernel can be seen as an infinite sum of Gaussian kernels. If one sets $\alpha = \alpha_0 / \beta$, then taking the limit $\beta \rightarrow \infty$ results in the Gaussian kernel with scaling parameter $\alpha_0$.

source
MLKernels.GammaRationalKernel — Type.
GammaRationalKernel([α [,β [,γ]]])

The gamma-rational kernel is a generalization of the rational-quadratic kernel with an additional shape parameter:

\[\kappa(\mathbf{x},\mathbf{y}) = \left(1 +\alpha ||\mathbf{x},\mathbf{y}||^{\gamma}\right)^{-\beta} \qquad \alpha > 0, \; \beta > 0, \; 0 < \gamma \leq 1\]

where $\alpha$ is a scaling parameter and $\beta$ and $\gamma$ are shape parameters.

source
MLKernels.MaternKernel — Type.
MaternKernel([ν=1 [,θ=1]])

The Matern kernel is a Mercer kernel given by:

\[\kappa(\mathbf{x},\mathbf{y}) = \frac{1}{2^{\nu-1}\Gamma(\nu)} \left(\frac{\sqrt{2\nu}||\mathbf{x}-\mathbf{y}||}{\theta}\right)^{\nu} K_{\nu}\left(\frac{\sqrt{2\nu}||\mathbf{x}-\mathbf{y}||}{\theta}\right)\]

where $\Gamma$ is the gamma function, $K_{\nu}$ is the modified Bessel function of the second kind, $\nu > 0$ and $\theta > 0$.

source
MLKernels.LinearKernel — Type.
LinearKernel([a=1 [,c=1]])

The linear kernel is a Mercer kernel given by:

\[\kappa(\mathbf{x},\mathbf{y}) = a \mathbf{x}^\intercal \mathbf{y} + c \qquad \alpha > 0, \; c \geq 0\]
source
MLKernels.PolynomialKernel — Type.
PolynomialKernel([a=1 [,c=1 [,d=3]]])

The polynomial kernel is a Mercer kernel given by:

\[\kappa(\mathbf{x},\mathbf{y}) = (a \mathbf{x}^\intercal \mathbf{y} + c)^d \qquad \alpha > 0, \; c \geq 0, \; d \in \mathbb{Z}_{+}\]
source
MLKernels.ExponentiatedKernel — Type.
ExponentiatedKernel([a=1])

The exponentiated kernel is a Mercer kernel given by:

\[\kappa(\mathbf{x},\mathbf{y}) = \exp\left(a \mathbf{x}^\intercal \mathbf{y} \right) \qquad a > 0\]
source
MLKernels.PeriodicKernel — Type.
PeriodicKernel([α=1 [,p=π]])

The periodic kernel is given by:

\[\kappa(\mathbf{x},\mathbf{y}) = \exp\left(-\alpha \sum_{i=1}^n \sin(p(x_i - y_i))^2\right) \qquad p >0, \; \alpha > 0\]

where $\mathbf{x}$ and $\mathbf{y}$ are $n$ dimensional vectors. The parameters $p$ and $\alpha$ are scaling parameters for the periodicity and the magnitude, respectively. This kernel is useful when data has periodicity to it.

source

Negative Definite Kernels

MLKernels.PowerKernel — Type.
PowerKernel([γ=1])

The Power Kernel is a negative definite kernel given by:

\[\kappa(\mathbf{x},\mathbf{y}) = \|\mathbf{x} - \mathbf{y} \|^{2\gamma} \qquad \gamma \in (0,1]\]
source
MLKernels.LogKernel — Type.
LogKernel([α [,γ]])

The Log Kernel is a negative definite kernel given by:

\[\kappa(\mathbf{x},\mathbf{y}) = \log \left(1 + \alpha\|\mathbf{x} - \mathbf{y} \|^{2\gamma}\right) \qquad \alpha > 0, \; \gamma \in (0,1]\]
source

Other Kernels

MLKernels.SigmoidKernel — Type.
SigmoidKernel([a=1 [,c=1]])

The Sigmoid Kernel is given by

\[\kappa(\mathbf{x},\mathbf{y}) = \tanh(a \mathbf{x}^\intercal \mathbf{y} + c) \qquad \alpha > 0, \; c \geq 0\]

The sigmoid kernel is a not a true kernel, although it has been used in application.

source