# Sigmoid function

A sigmoid function , gooseneck function , Fermi function , or S function is a mathematical function with an S-shaped graph . Often the term sigmoid function is related to the special case logistic function that is defined by the equation

${\ displaystyle \ operatorname {sig} (t) = {\ frac {1} {1 + e ^ {- t}}} = {\ frac {e ^ {t}} {1 + e ^ {t}}} = {\ frac {1} {2}} \ cdot \ left (1+ \ tanh {\ frac {t} {2}} \ right)}$

is described. It is the Euler number . This special sigmoid function is essentially a scaled and shifted hyperbolic tangent function and has corresponding symmetries . ${\ displaystyle e}$

The inverse function of this function is:

${\ displaystyle \ operatorname {sig} ^ {- 1} (y) = - \ ln \ left ({\ frac {1} {y}} - 1 \ right) = \ ln \ left ({\ frac {y} {1-y}} \ right) = 2 \ cdot \ operatorname {artanh} (2 \ cdot y-1).}$

## Sigmoid functions in general

Comparison of some sigmoid functions. Here they are standardized in such a way that their limit values ​​are −1 or 1 and the slopes in 0 are equal to 1.

In general, a sigmoid function is a bounded and differentiable real function with a consistently positive or consistently negative first derivative and exactly one inflection point .

In addition to the logistic function, the set of sigmoid functions includes the arctangent , the hyperbolic tangent and the error function , all of which are transcendent , but also simple algebraic functions such as . The integral of every continuous , positive function with a "mountain" (more precisely: with exactly one local maximum and no local minimum, e.g. the Gaussian bell curve ) is also a sigmoid function. Therefore, many cumulative distribution functions are sigmoidal. ${\ displaystyle f (x) = {\ tfrac {x} {\ sqrt {1 + x ^ {2}}}}}$

## Sigmoid functions in neural networks

Sigmoid functions are often used as activation functions in artificial neural networks , since the use of differentiable functions enables learning mechanisms such as the backpropagation algorithm to be used. As the activation function of an artificial neuron , the sigmoid function is applied to the sum of the weighted input values ​​to obtain the output of the neuron.

The sigmoid function is preferred as an activation function due to its simple differentiability, because the following applies to the logistic function:

${\ displaystyle \ operatorname {sig} ^ {\ prime} (t) = \ operatorname {sig} (t) \ left (1- \ operatorname {sig} (t) \ right).}$

The following applies to the derivation of the sigmoid function hyperbolic tangent:

${\ displaystyle \ tanh ^ {\ prime} (t) = \ left (1+ \ tanh (t) \ right) \ left (1- \ tanh (t) \ right).}$

## Efficient calculation

With Unums of type III, the logistic function given above can be approximately efficiently calculated by elegantly using the representation of the floating point number input.