Sigmoid function

The logistic curve

A sigmoid function , gooseneck function , Fermi function , or S function is a mathematical function with an S-shaped graph . Often the term sigmoid function is related to the special case logistic function that is defined by the equation

${\displaystyle \operatorname {sig} (t)={\frac {1}{1+e^{-t}}}={\frac {e^{t}}{1+e^{t}}}={\frac {1}{2}}\cdot \left(1+\tanh {\frac {t}{2}}\right)}$

is described. It is the Euler number . This special sigmoid function is essentially a scaled and shifted hyperbolic tangent function and has corresponding symmetries . ${\displaystyle e}$

The inverse function of this function is:

${\displaystyle \operatorname {sig} ^{-1}(y)=-\ln \left({\frac {1}{y}}-1\right)=\ln \left({\frac {y}{1-y}}\right)=2\cdot \operatorname {artanh} (2\cdot y-1).}$

Sigmoid functions in general

Comparison of some sigmoid functions. Here they are standardized in such a way that their limit values ​​are −1 or 1 and the slopes in 0 are equal to 1.

In general, a sigmoid function is a bounded and differentiable real function with a consistently positive or consistently negative first derivative and exactly one inflection point .

In addition to the logistic function, the set of sigmoid functions includes the arctangent , the hyperbolic tangent and the error function , all of which are transcendent , but also simple algebraic functions such as . The integral of every continuous , positive function with a "mountain" (more precisely: with exactly one local maximum and no local minimum, e.g. the Gaussian bell curve ) is also a sigmoid function. Therefore, many cumulative distribution functions are sigmoidal. ${\displaystyle f(x)={\tfrac {x}{\sqrt {1+x^{2}}}}}$

Sigmoid functions in neural networks

Sigmoid functions are often used as activation functions in artificial neural networks , since the use of differentiable functions enables learning mechanisms such as the backpropagation algorithm to be used. As the activation function of an artificial neuron , the sigmoid function is applied to the sum of the weighted input values ​​to obtain the output of the neuron.

The sigmoid function is preferred as an activation function due to its simple differentiability, because the following applies to the logistic function:

${\displaystyle \operatorname {sig} ^{\prime }(t)=\operatorname {sig} (t)\left(1-\operatorname {sig} (t)\right).}$

The following applies to the derivation of the sigmoid function hyperbolic tangent:

${\displaystyle \tanh ^{\prime }(t)=\left(1+\tanh(t)\right)\left(1-\tanh(t)\right).}$

Efficient calculation

With Unums of type III, the logistic function given above can be approximately efficiently calculated by elegantly using the representation of the floating point number input.

See also

• Logistic distribution
• Artificial neural network
• Fermi-Dirac statistics
• Gompertz function

Web links

• Eric W. Weisstein : Sigmoid Function . In: MathWorld (English).

Individual evidence

1. ↑ Single Neuron ::: Neural Networks. Retrieved April 4, 2019 . 
2. ^ John L. Gustafson, Isaac Yonemoto: Beating Floating Point at its Own Game: Posit Arithmetic. (PDF) June 12, 2017, accessed on December 28, 2019 .