Kernels

Kernels

KernelMercerNegative DefiniteStationaryIsotropic
Exponential Kernel
Rational Quadratic Kernel
Exponentiated Kernel

Exponential Kernel

Exponential Kernel

The exponential kernel (see ExponentialKernel) is an isotropic Mercer kernel of the form:

\[\kappa(\mathbf{x},\mathbf{y}) = \exp\left(-\alpha ||\mathbf{x} - \mathbf{y}||\right) \qquad \alpha > 0\]

where $\alpha$ is a positive scaling parameter of the Euclidean distance. This kernel may also be referred to as the Laplacian kernel (see LaplacianKernel).

Squared-Exponential Kernel

A similar form of the exponential kernel squares the Euclidean distance:

\[\kappa(\mathbf{x},\mathbf{y}) = \exp\left(-\alpha ||\mathbf{x} - \mathbf{y}||^2\right) \qquad \alpha > 0\]

In this case, the kernel is often referred to as the squared exponential kernel (see SquaredExponentialKernel) or the Gaussian kernel (see GaussianKernel).

$\gamma$-Exponential Kernel

Both the exponential and the squared exponential kernels are specific cases of the more general $\gamma$-exponential kernel:

\[\kappa(\mathbf{x},\mathbf{y}) = \exp\left(-\alpha ||\mathbf{x} - \mathbf{y}||^{2\gamma}\right) \qquad \alpha > 0, \;\; 0 < \gamma \leq 1\]

where $\gamma$ is an additional shape parameter of the Euclidean distance.

Interface

ExponentialKernel([α=1])

The exponential kernel is an isotropic Mercer kernel given by the formula:

    κ(x,y) = exp(-α‖x-y‖)   α > 0

where α is a positive scaling parameter. See also SquaredExponentialKernel for a related form of the kernel or GammaExponentialKernel for a generalization.

Examples

julia> ExponentialKernel()
ExponentialKernel{Float64}(1.0)

julia> ExponentialKernel(2.0f0)
ExponentialKernel{Float32}(2.0)
source
SquaredExponentialKernel([α=1])

The squared exponential kernel is an isotropic Mercer kernel given by the formula:

    κ(x,y) = exp(-α‖x-y‖²)   α > 0

where α is a positive scaling parameter. See also ExponentialKernel for a related form of the kernel or GammaExponentialKernel for a generalization.

Examples

julia> SquaredExponentialKernel()
SquaredExponentialKernel{Float64}(1.0)

julia> SquaredExponentialKernel(2.0f0)
SquaredExponentialKernel{Float32}(2.0)
source
GammaExponentialKernel([α=1 [,γ=1]])

The $\gamma$-exponential kernel is an isotropic Mercer kernel given by the formula:

    κ(x,y) = exp(-α‖x-y‖²ᵞ)   α > 0, γ ∈ (0,1]

where α is a scaling parameter and γ is a shape parameter of the Euclidean distance. When γ = 1 use SquaredExponentialKernel and SquaredExponentialKernel when γ = 0.5 since these are more efficient implementations.

Examples

julia> GammaExponentialKernel()
GammaExponentialKernel{Float64}(1.0,1.0)

julia> GammaExponentialKernel(2.0f0)
GammaExponentialKernel{Float32}(2.0,1.0)

julia> GammaExponentialKernel(2.0, 0.5)
GammaExponentialKernel{Float64}(2.0,0.5)
source
LaplacianKernel([α=1])

Alias for ExponentialKernel.

source
GaussianKernel([α=1])

Alias of SquaredExponentialKernel.

source
RadialBasisKernel([σ=1])

Create a SquaredExponentialKernel using the following convention for Radial Basis Function Kernel.

κ(x,y) = exp(-‖x-y‖²/σ²)
source

Rational-Quadratic Kernel

Rational-Quadratic Kernel

The rational-quadratic kernel (see RationalQuadraticKernel) is an isotropic Mercer kernel given by the formula:

\[\kappa(\mathbf{x},\mathbf{y}) = \left(1 +\alpha ||\mathbf{x} - \mathbf{y}||^{2}\right)^{-\beta} \qquad \alpha > 0, \;\; \beta > 0\]

where $\alpha$ is a positive scaling parameter and $\beta$ is a shape parameter of the Euclidean distance.

$\gamma$-Rational-Quadratic Kernel

The rational-quadratic kernel is a special case with $\gamma = 1$ of the more general $\gamma$-rational-quadratic kernel (see GammaRationalQuadraticKernel):

\[\kappa(\mathbf{x},\mathbf{y}) = \left(1 +\alpha ||\mathbf{x} - \mathbf{y}||^{2\gamma}\right)^{-\beta} \qquad \alpha > 0, \; \beta > 0, \; 0 < \gamma \leq 1\]

where $\alpha$ is a positive scaling parameter, $\beta$ is a positive shape parameter and $\gamma$ is a shape parameter of the Euclidean distance.

Interface

RationalQuadraticKernel([α [,β]])

The rational quadratic kernel is an isotropic Mercer kernel given by the formula:

    κ(x,y) = (1 + α‖x-y‖²)⁻ᵝ   α > 0, β > 0

where α is a scaling parameter and β is a shape parameter. The rational quadratic kernel is a special of the more general gamma-rational-quadratic kernel (see GammaRationalQuadraticKernel) with γ = 1.

Examples

julia> RationalQuadraticKernel()
RationalQuadraticKernel{Float64}(1.0,1.0)

julia> RationalQuadraticKernel(2.0f0)
RationalQuadraticKernel{Float32}(2.0,1.0)

julia> RationalQuadraticKernel(2.0f0, 2.0)
RationalQuadraticKernel{Float64}(2.0,2.0)
source
GammaRationalKernel([α [,β [,γ]]])

The gamma-rational-quadratic kernel is a generalization of the rational-quadratic kernel with an additional shape parameter γ:

    κ(x,y) = (1 + α‖x-y‖²ᵞ)⁻ᵝ   α > 0, β > 0, γ ∈ (0,1]

where $\alpha$ is a scaling parameter, $\beta$ is a shape parameter and $\gamma$ is a shape parameter of the Euclidean distance.

Examples

julia> GammaRationalQuadraticKernel()
GammaRationalQuadraticKernel{Float64}(1.0,1.0,1.0)

julia> GammaRationalQuadraticKernel(2.0f0)
GammaRationalQuadraticKernel{Float32}(2.0,1.0,1.0)

julia> GammaRationalQuadraticKernel(2.0f0, 2.0f0)
GammaRationalQuadraticKernel{Float32}(2.0,2.0,1.0)

julia> GammaRationalQuadraticKernel(2.0f0, 2.0f0, 0.5f0)
GammaRationalQuadraticKernel{Float32}(2.0,2.0,0.5)
source

Exponentiated Kernel

The exponentiated kernel (see ExponentiatedKernel) is a Mercer kernel given by:

\[\kappa(\mathbf{x},\mathbf{y}) = \exp\left(a \mathbf{x}^\intercal \mathbf{y} \right) \qquad a > 0\]

where $\alpha$ is a positive shape parameter.

Interface

ExponentiatedKernel([α=1])

The exponentiated kernel is a Mercer kernel given by:

    κ(x,y) = exp(α⋅xᵀy)   α > 0

where α is a positive scaling parameter.

Examples

julia> ExponentiatedKernel()
ExponentiatedKernel{Float64}(1.0)

julia> ExponentiatedKernel(2)
ExponentiatedKernel{Float64}(2.0)

julia> ExponentiatedKernel(2.0f0)
ExponentiatedKernel{Float32}(2.0)
source

Matern Kernel

The Matern kernel is a Mercer kernel given by:

\[\kappa(\mathbf{x},\mathbf{y}) = \frac{1}{2^{\nu-1}\Gamma(\nu)} \left(\frac{\sqrt{2\nu}||\mathbf{x}-\mathbf{y}||}{\theta}\right)^{\nu} K_{\nu}\left(\frac{\sqrt{2\nu}||\mathbf{x}-\mathbf{y}||}{\theta}\right)\]

where $\nu$ and $\rho$ are positive shape parameters.

Interface

MaternKernel([ν=1 [, θ=1]])

The Matern kernel is a Mercer kernel with parameters ν > 0 and ρ > 0. See the published documentation for the full definition of the function.

Examples

julia> MaternKernel()
MaternKernel{Float64}(1.0,1.0)

julia> MaternKernel(2.0f0)
MaternKernel{Float32}(2.0,1.0)

julia> MaternKernel(2.0f0, 2.0)
MaternKernel{Float64}(2.0,2.0)
source

Polynomial Kernel

The polynomial kernel is a Mercer kernel given by:

\[\kappa(\mathbf{x},\mathbf{y}) = (a \mathbf{x}^\intercal \mathbf{y} + c)^d \qquad \alpha > 0, \; c \geq 0, \; d \in \mathbb{Z}_{+}\]

where $a$ is a positive scale parameter, $c$ is a non-negative shape parameter and $d$ is a shape parameter that determines the degree of the resulting polynomial.

Interface

PolynomialKernel([a=1 [,c=1 [,d=3]]])

The polynomial kernel is a Mercer kernel given by:

    κ(x,y) = (a⋅xᵀy + c)ᵈ   α > 0, c ≧ 0, d ∈ ℤ⁺

Examples

julia> PolynomialKernel(2.0f0)
PolynomialKernel{Float32}(2.0,1.0,3)

julia> PolynomialKernel(2.0f0, 2.0)
PolynomialKernel{Float64}(2.0,2.0,3)

julia> PolynomialKernel(2.0f0, 2.0, 2)
PolynomialKernel{Float64}(2.0,2.0,2)
source

Periodic Kernel

The periodic kernel is given by:

\[\kappa(\mathbf{x},\mathbf{y}) = \exp\left(-\alpha \sum_{i=1}^n \sin(x_i - y_i)^2\right) \qquad \alpha > 0\]

where $a$ is a positive scale parameter.

Interface

PeriodicKernel

Power Kernel

The power kernel is given by:

\[\kappa(\mathbf{x},\mathbf{y}) = \|\mathbf{x} - \mathbf{y} \|^{2\gamma} \qquad \gamma \in (0,1]\]

where $\gamma$ is a shape parameter of the Euclidean distance.

Interface

PowerKernel([γ=1])

The Power Kernel is a negative definite kernel given by:

    κ(x,y) = ‖x-y‖²ᵞ   γ ∈ (0,1]

where γ is a shape parameter of the Euclidean distance.

Examples

julia> PowerKernel()
PowerKernel{Float64}(1.0)

julia> PowerKernel(0.5f0)
PowerKernel{Float32}(0.5)
source

Log Kernel

The log kernel is a negative definite kernel given by:

\[\kappa(\mathbf{x},\mathbf{y}) = \log \left(1 + \alpha\|\mathbf{x} - \mathbf{y} \|^{2\gamma}\right) \qquad \alpha > 0, \; \gamma \in (0,1]\]

where $\alpha$ is a positive scaling parameter and $\gamma$ is a shape parameter.

Interface

LogKernel([α [,γ]])

The Log Kernel is a negative definite kernel given by the formula:

    κ(x,y) = log(1 + α‖x-y‖²ᵞ)   α > 0, γ ∈ (0,1]

where α is a scaling parameter and γ is a shape parameter of the Euclidean distance.

Examples

julia> LogKernel()
LogKernel{Float64}(1.0,1.0)

julia> LogKernel(0.5f0)
LogKernel{Float32}(0.5,1.0)

julia> LogKernel(0.5, 0.5)
LogKernel{Float64}(0.5,0.5)
source

Sigmoid Kernel

The Sigmoid Kernel is given by:

\[\kappa(\mathbf{x},\mathbf{y}) = \tanh(a \mathbf{x}^\intercal \mathbf{y} + c) \qquad \alpha > 0, \; c \geq 0\]

The sigmoid kernel is a not a true kernel, although it has been used in application.

SigmoidKernel([a=1 [,c=1]])

The Sigmoid Kernel is given by:

    κ(x,y) = tanh(a⋅xᵀy + c)

Examples

julia> SigmoidKernel()
SigmoidKernel{Float64}(1.0,1.0)

julia> SigmoidKernel(0.5f0)
SigmoidKernel{Float32}(0.5,1.0)

julia> SigmoidKernel(0.5f0, 0.5)
SigmoidKernel{Float64}(0.5,0.5)
source