Sigmoid function

from Wikipedia, the free encyclopedia

A sigmoid function , gooseneck function , Fermi function , or S function is a mathematical function with an S-shaped graph . Often the term sigmoid function is related to the special case logistic function that is defined by the equation

is described. It is the Euler number . This special sigmoid function is essentially a scaled and shifted hyperbolic tangent function and has corresponding symmetries .

The inverse function of this function is:

Sigmoid functions in general

Comparison of some sigmoid functions. Here they are standardized in such a way that their limit values ​​are −1 or 1 and the slopes in 0 are equal to 1.

In general, a sigmoid function is a bounded and differentiable real function with a consistently positive or consistently negative first derivative and exactly one inflection point .

In addition to the logistic function, the set of sigmoid functions includes the arctangent , the hyperbolic tangent and the error function , all of which are transcendent , but also simple algebraic functions such as . The integral of every continuous , positive function with a "mountain" (more precisely: with exactly one local maximum and no local minimum, e.g. the Gaussian bell curve ) is also a sigmoid function. Therefore, many cumulative distribution functions are sigmoidal.

Sigmoid functions in neural networks

Sigmoid functions are often used as activation functions in artificial neural networks , since the use of differentiable functions enables learning mechanisms such as the backpropagation algorithm to be used. As the activation function of an artificial neuron , the sigmoid function is applied to the sum of the weighted input values ​​to obtain the output of the neuron.

The sigmoid function is preferred as an activation function due to its simple differentiability, because the following applies to the logistic function:

The following applies to the derivation of the sigmoid function hyperbolic tangent:

Efficient calculation

With Unums of type III, the logistic function given above can be approximately efficiently calculated by elegantly using the representation of the floating point number input.

See also

Web links

Individual evidence

  1. Single Neuron ::: Neural Networks. Retrieved April 4, 2019 .
  2. ^ John L. Gustafson, Isaac Yonemoto: Beating Floating Point at its Own Game: Posit Arithmetic. (PDF) June 12, 2017, accessed on December 28, 2019 .