Moment generating function

The moment generating function is a function that is assigned to a random variable in probability theory . In many cases this function is defined in the vicinity of the zero point in the real or complex numbers and can then be used to calculate the moments of the random variables by means of a derivative , which explains its name.

definition

The moment generating function of a random variable is defined by ${\displaystyle X}$

${\displaystyle M_{X}(t):=E\left(e^{tX}\right)}$,

where can be used for real or complex numbers, provided that the expected value exists on the right-hand side. This expression is at least defined for. In many cases, see below, this function is defined in a neighborhood of 0, and can then be expanded into a power series as follows . ${\displaystyle t}$ ${\displaystyle t=0}$

${\displaystyle M_{X}(t)=E\left(\sum _{n=0}^{\infty }{\frac {(tX)^{n}}{n!}}\right)=\sum _{n=0}^{\infty }{\frac {t^{n}}{n!}}E(X^{n})=\sum _{n=0}^{\infty }{\frac {t^{n}}{n!}}m_{X}^{n}}$.

The following applies and they are the moments of . ${\displaystyle 0^{0}:=1}$ ${\displaystyle m_{X}^{n}=E(X^{n})}$ ${\displaystyle X}$

The moment generating function depends only on the distribution of . If the moment-generating function of a distribution exists in a neighborhood of 0, it is said, somewhat imprecise but commonly used, that the distribution has a moment-generating function. If only exists for , then we say accordingly that the distribution has no moment-generating function. ${\displaystyle X}$ ${\displaystyle M_{X}(t)}$ ${\displaystyle t=0}$

Continuous probability distributions

If there is a continuous probability density , one can write the above expected value using this density and obtain for the moment generating function ${\displaystyle X}$ ${\displaystyle f}$

${\displaystyle M_{X}(t)=\int _{-\infty }^{\infty }e^{tx}f(x)\,\mathrm {d} x}$

${\displaystyle =\int _{-\infty }^{\infty }\left(1+tx+{\frac {t^{2}}{2!}}x^{2}+\dotsb \right)f(x)\,\mathrm {d} x}$

${\displaystyle =1+tm_{X}^{1}+{\frac {t^{2}}{2!}}m_{X}^{2}+\dotsb }$

given, where the -th moment is of. The expression is therefore precisely the two-sided Laplace transformation of the probability measure established by. ${\displaystyle m_{X}^{k}}$ ${\displaystyle k}$ ${\displaystyle X}$ ${\displaystyle M_{X}\left(-t\right)}$ ${\displaystyle X}$

Remarks

Origin of the concept of the moment generating function

The term generating moment refers to the fact that the -th derivative of at point 0 (zero) is equal to the -th moment of the random variable : ${\displaystyle k}$ ${\displaystyle M_{X}}$ ${\displaystyle k}$ ${\displaystyle X}$

${\displaystyle {\frac {\mathrm {d} ^{k}}{\mathrm {d} t^{k}}}M_{X}(t){\biggr \vert }_{t=0}=E(X^{k})=m_{X}^{k}}$.

This can be read directly from the power series given above. By specifying all non-vanishing moments, every probability distribution is completely determined if the moment-generating function exists on an open interval . ${\displaystyle (-\varepsilon ,\varepsilon )}$ ${\displaystyle (\varepsilon >0)}$

Relationship with the characteristic function

The torque generating function is closely related to the characteristic function . It applies if the torque generating function exists. In contrast to the moment generating function, the characteristic function exists for any random variable. ${\displaystyle \varphi _{X}(t)=E\left(e^{\mathrm {i} tX}\right)}$ ${\displaystyle \varphi _{X}(t)=M_{iX}(t)=M_{X}(\mathrm {i} t)}$

Relation to the probability generating function

There is also a connection to the probability-generating function . However, this is only defined for -value random variables as . This applies to discrete random variables. ${\displaystyle \mathbb {N} _{0}}$ ${\displaystyle m_{X}(t)=\operatorname {E} (t^{X})}$ ${\displaystyle m_{X}(e^{t})=M_{X}(t)}$

Relationship with the cumulant generating function

The cumulative generating function is defined as the natural logarithm of the moment generating function. The term cumulative is derived from it.

Sums of independent random variables

The moment-generating function of a sum of independent random variables is the product of its moment-generating functions: If are independent, then applies to ${\displaystyle X_{1},\dotsc ,X_{n}}$ ${\displaystyle Y=X_{1}+\dotsb +X_{n}}$

${\displaystyle M_{Y}(t)=E(e^{tY})=E(e^{tX_{1}+\ldots +tX_{n}})=E(e^{tX_{1}}\cdots e^{tX_{n}})=E(e^{tX_{1}})\cdots E(e^{tX_{n}})=M_{X_{1}}(t)\cdots M_{X_{n}}(t)}$,

with the penultimate equal sign that the expected value of a product of independent random variables is equal to the product of their expected values.

Uniqueness property

If the moment-generating function of a random variable in a neighborhood of finite, it determines the distribution of uniquely. Formally this means: ${\displaystyle X}$ ${\displaystyle 0}$ ${\displaystyle X}$

Let us be and two random variables with moment-generating functions and such that there is one with for everyone . Then applies if and only if applies to all . ${\displaystyle X}$ ${\displaystyle Y}$ ${\displaystyle M_{X}}$ ${\displaystyle M_{Y}}$ ${\displaystyle \varepsilon >0}$ ${\displaystyle M_{X}(s),M_{Y}(s)<\infty }$ ${\displaystyle s\in (-\varepsilon ,\varepsilon )}$ ${\displaystyle P_{X}=P_{Y}}$ ${\displaystyle M_{X}(s)=M_{Y}(s)}$ ${\displaystyle s\in (-\varepsilon ,\varepsilon )}$

Examples

For many distributions the moment generating function can be given directly:

distribution	Moment generating function M X (t)
Bernoulli distribution ${\displaystyle \mathrm {B} (p)}$	${\displaystyle M_{X}(t)=1-p+pe^{t}}$
Beta distribution ${\displaystyle \mathrm {B} (a,b,p,q)}$	${\displaystyle M_{X}(t)=1+\sum _{n=1}^{\infty }\left(\prod _{k=0}^{n-1}{\frac {a+k}{a+b+k}}\right){\frac {t^{n}}{n!}}}$
Binomial distribution ${\displaystyle \mathrm {B} (p,n)}$	${\displaystyle M_{X}(t)=(1-p+pe^{t})^{n}}$
Cauchy distribution	The Cauchy distribution has no moment-generating function.
Chi-square distribution ${\displaystyle \chi _{n}^{2}}$	${\displaystyle M_{X}(t)={\frac {1}{(1-2t)^{n/2}}}}$
Erlang distribution ${\displaystyle \mathrm {Erlang} (\lambda ,n)}$	${\displaystyle M_{X}(t)=\left({\frac {\lambda }{\lambda -t}}\right)^{n}}$ For ${\displaystyle t<\lambda }$
Exponential distribution ${\displaystyle \mathrm {Exp} (\lambda )}$	${\displaystyle M_{X}(t)={\frac {\lambda }{\lambda -t}}}$ For ${\displaystyle t<\lambda }$
Gamma distribution ${\displaystyle \gamma (p,b)}$	${\displaystyle M_{X}(t)=\left({\frac {b}{b-t}}\right)^{p}}$
Geometric distribution with parameters ${\displaystyle p}$	${\displaystyle M_{X}(t)={\frac {pe^{t}}{1-(1-p)e^{t}}}}$
Even distribution over ${\displaystyle [0,a]}$	${\displaystyle M_{X}(t)={\frac {e^{ta}-1}{ta}}}$
Laplace distribution with parameters ${\displaystyle \mu ,\sigma }$	${\displaystyle M_{X}(t)={\frac {e^{\mu t}}{1-\sigma ^{2}t^{2}}}}$
Negative binomial distribution ${\displaystyle \mathrm {NB} (r,p)}$	${\displaystyle M_{X}(t)=\left({\frac {pe^{t}}{1-(1-p)e^{t}}}\right)^{r}}$ For ${\displaystyle t<|\ln(1-p)|}$
Normal distribution ${\displaystyle \mathrm {N} (\mu ,\sigma ^{2})}$	${\displaystyle M_{X}(t)=\exp {\left(\mu t+{\frac {\sigma ^{2}t^{2}}{2}}\right)}}$
Poisson distribution with parameters ${\displaystyle \lambda }$	${\displaystyle M_{X}(t)=\exp(\lambda (e^{t}-1))}$

Generalization to multidimensional random variables

The moment-generating function can be extended to -dimensional real random vectors as follows: ${\displaystyle \ell }$ ${\displaystyle \mathbf {X} =(X_{1},\dotsc ,X_{\ell })}$

${\displaystyle M_{\mathbf {X} }(t)=M_{\mathbf {X} }(t_{1},\dots ,t_{l})=\operatorname {E} (e^{\langle t,\mathbf {X} \rangle })=\operatorname {E} \left(\prod _{j=1}^{\ell }e^{t_{j}X_{j}}\right)}$,

where denotes the standard scalar product. ${\displaystyle \langle t,\mathbf {X} \rangle =\sum \limits _{j=1}^{\ell }t_{j}X_{j}}$

If the components of the random vector are independent of each other in pairs, then the moment-generating function results as the product of the moment-generating functions of one-dimensional random variables:

${\displaystyle M_{\mathbf {X} }(t_{1},\dots ,t_{l})=\operatorname {E} (e^{\langle t,\mathbf {X} \rangle })=\prod _{j=1}^{\ell }\operatorname {E} \left(e^{t_{j}X_{j}}\right)=\prod _{j=1}^{\ell }M_{X_{j}}(t_{j})}$.

literature

• Klaus D. Schmidt: Measure and Probability. Springer, Berlin / Heidelberg 2009, ISBN 978-3-540-89729-3 , p. 378 ff.

Individual evidence

1. ^ Robert G. Gallager: Stochastic Processes. Cambridge University Press, 2013, ISBN 978-1-107-03975-9 , Chapter 1.5.5: Moment generating functions and other transforms
2. ^ JH Curtiss: A Note on the Theory of Moment Generating Functions. In: The Annals of Mathematical Statistics. Volume 13, No. 4, 1942, ISSN  0003-4851 , pp. 430–433, accessed December 30, 2012, (PDF; 402 kB).
3. ↑ Otto JWF Kardaun: Classical Methods of Statistics. Springer-Verlag, 2005, ISBN 3-540-21115-2 , p. 44.
4. ^ Allan Gut: Probability: A Graduate Course. Springer-Verlag, 2012, ISBN 978-1-4614-4707-8 , chapter 8, example 8.2.
5. ^ AC Davison: Statistical Models. Cambridge University Press, 2008, ISBN 978-1-4672-0331-9 , Chapter 3.2.
6. ^ Hisashi Tanizaki: Computational Methods in Statistics and Econometrics. Taylor and Francis Publisher, 2004, ISBN 0-203-02202-5 , Section 2.2.11.