Convolution (stochastics)

This article deals with convolution in stochastics. For other meanings, see Convolution (math) and Convolution

In stochastics, convolution is an operation that combines two probability measures into a new probability measure. It makes it possible to assign a meaningful probability to the sum of these values ​​for values ​​that are subject to chance. The distribution of the sum of two independent random variables is exactly the convolution of the distribution of the individual random variables.

If the probability measures under consideration have a probability function or a probability density function , the convolution of the probability measures can be traced back to the convolution (of functions) of the probability functions or probability density functions.

Probability measures on the whole numbers

definition

Given are two discrete probability measures on the whole numbers with probability functions and . The convolution of the probability measures and is then the probability measure on which the probability function ${\displaystyle P,Q}$ ${\displaystyle \mathbb {Z} }$ ${\displaystyle f_{P}}$ ${\displaystyle f_{Q}}$ ${\displaystyle P*Q}$ ${\displaystyle P}$ ${\displaystyle Q}$ ${\displaystyle \mathbb {Z} }$

${\displaystyle f_{P*Q}(k):=\sum _{(i,j)\in \mathbb {Z} ^{2} \atop i+j=k}f_{P}(i)f_{Q}(j)=\sum _{i=-\infty }^{\infty }f_{P}(i)f_{Q}(k-i)}$

owns. So it is

${\displaystyle f_{P*Q}=f_{P}*f_{Q}}$,

where the convolution of the functions and denotes. ${\displaystyle f_{P}*f_{Q}}$ ${\displaystyle f_{P}}$ ${\displaystyle f_{Q}}$

comment

If the probability functions are only defined on a subset of the integers, such as or , then they are continued outside these sets by the value zero, i.e. with . For the special case that both probability measures are defined on the natural numbers, then the following applies to the convolution ${\displaystyle \mathbb {N} }$ ${\displaystyle \{0,1,\dots ,n\}}$ ${\displaystyle f(i)=0}$

${\displaystyle f_{P*Q}(k)=\sum _{i=0}^{k}f_{P}(i)f_{Q}(k-i)}$.

Furthermore, the convolution is uniquely determined by specifying the probability functions, since a probability measure is uniquely determined on a discrete probability space by specifying the probability function.

example

Let it be the Bernoulli distribution for the parameter , i.e. with a probability function ${\displaystyle P}$ ${\displaystyle p}$

${\displaystyle f_{P}(0)=1-p{\text{ und }}f_{P}(1)=p}$

and the binomial distribution for the parameters 2 and , i.e. with a probability function ${\displaystyle Q}$ ${\displaystyle p}$

${\displaystyle f_{Q}(j)={\binom {2}{j}}p^{j}(1-p)^{2-j}}$

for . ${\displaystyle j=0,1,2}$

In order to determine the probability function of the convolution at the point , one now creates all pairs for which applies and for which both and are not equal to zero. In the given case these are: ${\displaystyle k}$ ${\displaystyle (i,j)}$ ${\displaystyle i+j=k}$ ${\displaystyle f_{P}(i)}$ ${\displaystyle f_{Q}(j)}$

${\displaystyle {\begin{aligned}k=0:&\quad (i,j)=(0,0)\\k=1:&\quad (i,j)=(0,1);(1,0)\\k=2:&\quad (i,j)=(0,2);(1,1)\\k=3:&\quad (i,j)=(1,2)\end{aligned}}}$

Now you create the product of the corresponding one for each and add it up: For is therefore ${\displaystyle k}$ ${\displaystyle f_{P}(i)\cdot f_{Q}(j)}$ ${\displaystyle (i,j)}$ ${\displaystyle k=0}$

${\displaystyle f_{P*Q}(0)=f_{P}(0)f_{Q}(0)=(1-p)^{3}}$.

For the other values ​​then follows

${\displaystyle f_{P*Q}(1)=f_{P}(0)f_{Q}(1)+f_{P}(1)f_{Q}(0)=3p(1-p)^{2}}$

${\displaystyle f_{P*Q}(2)=f_{P}(0)f_{Q}(2)+f_{P}(1)f_{Q}(1)=3p^{2}(1-p)}$

${\displaystyle f_{P*Q}(3)=f_{P}(1)f_{Q}(2)=p^{3}}$

This is the probability function of a binomial distribution for the parameters 3 and , thus applies ${\displaystyle p}$

${\displaystyle \operatorname {Ber} (p)*\operatorname {Bin} (2,p)=\operatorname {Bin} (3,p)}$.

A closed representation of the probability function can also be derived from the direct convolution of the probability functions.

Continuous probability measures on the real numbers

definition

Given are two probability measures on the real numbers, provided with Borel's σ-algebra . and also have probability density functions and . ${\displaystyle P,Q}$ ${\displaystyle P}$ ${\displaystyle Q}$ ${\displaystyle f_{P}}$ ${\displaystyle f_{Q}}$

Then that probability measure is called with the probability density function ${\displaystyle \mathbb {R} }$

${\displaystyle f_{P*Q}(z)=\int _{\mathbb {R} }f_{P}(x)f_{Q}(z-x)\mathrm {d} \lambda (x)}$

the convolution of the probability measures and and is denoted by. The Lebesgue integral can often be replaced by a Riemann integral, in which case one writes instead of . ${\displaystyle P}$ ${\displaystyle Q}$ ${\displaystyle P*Q}$ ${\displaystyle \mathrm {d} x}$ ${\displaystyle \mathrm {d} \lambda (x)}$

It then applies

${\displaystyle f_{P*Q}=f_{P}*f_{Q}}$,

where the convolution of the functions and denotes. ${\displaystyle f_{P}*f_{Q}}$ ${\displaystyle f_{P}}$ ${\displaystyle f_{Q}}$

comment

Convolution is also defined for probability distributions on real numbers that do not have a probability density function (such as the Cantor distribution ). It is then given by the general case given below.

An important exception to this is the convolution with the Dirac distribution : if the probability density function has , then the probability density function has . ${\displaystyle \delta _{a}}$ ${\displaystyle P}$ ${\displaystyle f_{P}(x)}$ ${\displaystyle \delta _{a}*P}$ ${\displaystyle f_{\delta _{a}*P}(x)=f_{P}(x-a)}$

example

Let exponential distributions be for the identical parameter , i.e. with probability density function ${\displaystyle P,Q}$ ${\displaystyle \lambda }$

${\displaystyle f_{P}(x)=f_{Q}(x)=\lambda \exp(-\lambda x)\mathbf {1} _{x\geq 0}}$

Thereby the indicator function is on the crowd . Then applies to ${\displaystyle \mathbf {1} _{A}}$ ${\displaystyle A}$ ${\displaystyle z\geq 0}$

${\displaystyle {\begin{aligned}f_{P*Q}(z)&=\int _{\mathbb {R} }\lambda \exp(-\lambda x)\mathbf {1} _{x\geq 0}\lambda \exp(-\lambda (z-x))\mathbf {1} _{z-x\geq 0}\mathrm {d} x\\&=\lambda ^{2}\int _{0}^{\infty }\exp(-\lambda x-\lambda (z-x))\mathbf {1} _{z\geq x}\mathrm {d} x\\&=\lambda ^{2}\int _{0}^{z}\exp(-\lambda z)\mathrm {d} x\\&=\lambda ^{2}z\exp(-\lambda z).\end{aligned}}}$

This is the probability density function of an Erlang distribution or a gamma distribution for the parameters 2 and . The convolution of two exponential distributions thus results in an Erl or a gamma distribution. ${\displaystyle \lambda }$

General case

definition

Let be a set on which at least addition is explained. Let be a σ-algebra and the product σ-algebra on . Furthermore, two probability measures are given and the corresponding product measure . ${\displaystyle \Omega }$ ${\displaystyle {\mathcal {A}}}$ ${\displaystyle {\mathcal {A}}\otimes {\mathcal {A}}}$ ${\displaystyle \Omega \times \Omega }$ ${\displaystyle P_{1},P_{2}}$ ${\displaystyle (\Omega ,{\mathcal {A}})}$ ${\displaystyle P_{1}\otimes P_{2}}$

Then is the picture

${\displaystyle A:\Omega \times \Omega \to \Omega }$

defined by

${\displaystyle (x,y)\mapsto x+y}$

a - - measurable function (and thus a random variable ), then the image measure of under (or the distribution of the random variable ) is called the convolution of the probability measures and . So is ${\displaystyle {\mathcal {A}}\otimes {\mathcal {A}}}$ ${\displaystyle {\mathcal {A}}}$ ${\displaystyle P_{1}\otimes P_{2}}$ ${\displaystyle A}$ ${\displaystyle A}$ ${\displaystyle P_{1}}$ ${\displaystyle P_{2}}$

${\displaystyle P_{1}*P_{2}:=(P_{1}\otimes P_{2})\circ A^{-1}}$

or analog

${\displaystyle (P_{1}*P_{2})(B)=(P_{1}\otimes P_{2})(\{(x,y)\in \Omega \times \Omega \,|\,x+y\in B\})}$.

The above measurability conditions are always fulfilled, for example, if is a topological vector space and the Borel σ-algebra . This is especially the case when and . ${\displaystyle \Omega }$ ${\displaystyle {\mathcal {A}}}$ ${\displaystyle \Omega =\mathbb {R} ^{d}}$ ${\displaystyle {\mathcal {A}}={\mathcal {B}}(\mathbb {R} ^{d})}$

Derivation of the above special cases

For probability measures on it is sufficient to show the statement for the sets , since these form a generator of the σ-algebra (here the power set ). It is ${\displaystyle \mathbb {Z} }$ ${\displaystyle \{k\}}$

${\displaystyle {\begin{aligned}(P*Q)(\{k\})&=(P\otimes Q)(A^{-1}(\{k\}))=(P\otimes Q)(\{(i,j)\in \mathbb {Z} ^{2}\,|\,i+j=k\})\\&=\sum _{(i,j)\in \mathbb {Z} ^{2} \atop i+j=k}(P\otimes Q)(\{(i,j)\})=\sum _{(i,j)\in \mathbb {Z} ^{2} \atop i+j=k}P(\{i\})Q(\{j\})\\&=\sum _{(i,j)\in \mathbb {Z} ^{2} \atop i+j=k}f_{P}(i)f_{Q}(j)\end{aligned}}}$.

The first two steps are reformulations of the image measures of the distributions, the third follows from the σ-additivity and the disjunctivity of the , the fourth from the definition of the product measure and the last from the unambiguous characterization of the probability measures by their probability functions. ${\displaystyle \{(i,j)\}}$

Thus, the probability function given in the above section is the probability function of the folded probability measures , so the definitions match. ${\displaystyle f_{P*Q}}$ ${\displaystyle P*Q}$

Analogously follows for probability measures ${\displaystyle \mathbb {R} }$

${\displaystyle {\begin{aligned}(P*Q)((-\infty ,c])&=\iint _{x+y\leq c}f_{P}(x)f_{Q}(y)\mathrm {d} \lambda (x)\otimes \lambda (y)=\int _{\mathbb {R} }f_{P}(x)\int f_{Q}(y-x)\mathbf {1} _{y\leq c}\mathrm {d} \lambda (y)\mathrm {d} \lambda (x)\\&=\int _{-\infty }^{c}\int _{\mathbb {R} }f_{P}(x)f_{Q}(y-x)\mathrm {d} \lambda (x)\mathrm {d} \lambda (y)\end{aligned}}}$

by substitution and Fubini's theorem .

properties

Sum of independent random variables

An important property of the convolution of probability measures is that it can be used to determine the distribution of the sum of stochastically independent random variables . If and are stochastically independent random variables with distributions and , then the distribution of the sum of the random variables is the convolution of the distributions of the random variables, i.e. ${\displaystyle X}$ ${\displaystyle Y}$ ${\displaystyle P_{X}}$ ${\displaystyle P_{Y}}$

${\displaystyle P_{X+Y}=P_{X}*P_{Y}}$.

This central property follows directly from the definition of the folding as a measure of the image of addition. The stochastic independence of the construction follows from the product dimensions.

Probability-Generating, Moment-Generating and Characteristic Functions

For probability measures on , the convolution can be related to the probability-generating functions . It then applies ${\displaystyle P,Q}$ ${\displaystyle \mathbb {N} }$ ${\displaystyle m_{P},m_{Q}}$

${\displaystyle m_{P*Q}(t)=m_{P}(t)\cdot m_{Q}(t)}$.

The probability-generating function of the convolution of two probability measures is therefore the product of the probability-generating functions of the measures.

The same applies to the torque generating function and the characteristic function : ${\displaystyle M_{P}}$ ${\displaystyle \varphi _{P}}$

${\displaystyle M_{P*Q}(t)=M_{P}(t)\cdot M_{Q}(t)}$   and   ${\displaystyle \varphi _{P*Q}(t)=\varphi _{P}(t)\cdot \varphi _{Q}(t)}$

From this follow the addition identities for independent random variables:

${\displaystyle m_{X+Y}(t)=m_{X}(t)\cdot m_{Y}(t)}$

${\displaystyle M_{X+Y}(t)=M_{X}(t)\cdot M_{Y}(t)}$

${\displaystyle \varphi _{X+Y}(t)=\varphi _{X}(t)\cdot \varphi _{Y}(t)}$

Constructive terms

Convolutional semigroups

→ Main article : Convolutional semigroup

A convolution semigroup is a set of probability measures that is closed with respect to the convolution. This means that the convolution of two probability measures from the convolution half-group is again contained in the convolution half-group. Convolution half-groups occur, for example, in the investigation of characteristic functions or as an aid to the construction of stochastic processes with certain properties, such as the Wiener process . Examples of convolution semigroups are the binomial distributions for a fixed parameter or the Cauchy distribution. ${\displaystyle p}$

Infinite divisibility

→ Main article : Infinite divisibility

A probability measure is called infinitely divisible if there is another probability measure for each for which ${\displaystyle P}$ ${\displaystyle n\in \mathbb {N} }$ ${\displaystyle Q}$

${\displaystyle P=Q^{*n}}$

applies. Here referred to

${\displaystyle Q^{*n}:=\underbrace {Q*Q*\dots *Q} _{n{\text{ mal}}}}$

the n-fold successive execution of the convolution. can therefore always be represented as the nth folding power of a further probability measure. The equivalent formulation for distributions is that the distribution is always the sum of independent, identical distributed random variables . ${\displaystyle P}$ ${\displaystyle P}$ ${\displaystyle n}$

Folding identities

The following list contains important folding identities, but is by no means exhaustive. Further convolution identities can be found in the corresponding main articles on probability measures.

distribution	folding	Convolutional semigroup	Infinitely divisible
Discrete distributions
Bernoulli distribution ${\displaystyle \operatorname {Ber} (p)}$	${\displaystyle \operatorname {Ber} (p)*\operatorname {Ber} (p)=\operatorname {Bin} (2,p)}$	No	No
Binomial distribution ${\displaystyle \operatorname {Bin} (n,p)}$	${\displaystyle \operatorname {Bin} (n,p)*\operatorname {Bin} (m,p)=\operatorname {Bin} (n+m,p)}$	Yes on ${\displaystyle \mathbb {N} ^{+}}$	No
Poisson distribution ${\displaystyle \operatorname {Poi} (\lambda )}$	${\displaystyle \operatorname {Poi} (\lambda _{1})*\operatorname {Poi} (\lambda _{2})=\operatorname {Poi} (\lambda _{1}+\lambda _{2})}$	Yes on ${\displaystyle \mathbb {R} _{>0}}$	Yes, through ${\displaystyle \operatorname {Poi} (\lambda /n)}$
Geometric distribution ${\displaystyle \operatorname {Geom} (p)}$	${\displaystyle \operatorname {Geom} (p)*\operatorname {Geom} (p)=\operatorname {NegBin} (2,p)}$	No	Yes, through ${\displaystyle \operatorname {NegBin} (1/n,p)}$
Negative binomial distribution ${\displaystyle \operatorname {NegBin} (r,p)}$	${\displaystyle \operatorname {NegBin} (r,p)*\operatorname {NegBin} (s,p)=\operatorname {NegBin} (r+s,p)}$	Yes, depending on the definition on or on ${\displaystyle \mathbb {N} ^{+}}$ ${\displaystyle \mathbb {R} _{>0}}$	yes, through ${\displaystyle \operatorname {NegBin} (r/n,p)}$
Dirac distribution ${\displaystyle \delta _{x}}$	${\displaystyle \delta _{x}*\delta _{y}=\delta _{x+y}}$	On ${\displaystyle \mathbb {R} }$	Yes, through ${\displaystyle \delta _{x/n}}$
Absolutely continuous distributions
Standard normal distribution ${\displaystyle {\mathcal {N}}(0,1)}$	${\displaystyle {\mathcal {N}}(0,1)*{\mathcal {N}}(0,1)={\mathcal {N}}(0,2)}$	No	Yes, through ${\displaystyle {\mathcal {N}}(0,1/n)}$
Normal distribution ${\displaystyle {\mathcal {N}}(\mu ,\sigma ^{2})}$	${\displaystyle {\mathcal {N}}(\mu _{1},\sigma _{1}^{2})*{\mathcal {N}}(\mu _{2},\sigma _{2}^{2})={\mathcal {N}}(\mu _{1}+\mu _{2},\sigma _{1}^{2}+\sigma _{2}^{2})}$	On ${\displaystyle \mathbb {R} \times \mathbb {R} _{>0}}$	Yes, through ${\displaystyle {\mathcal {N}}(\mu /n,\sigma ^{2}/n)}$
Cauchy distribution ${\displaystyle \operatorname {Cau} (a)}$	${\displaystyle \operatorname {Cau} (a)*\operatorname {Cau} (b)=\operatorname {Cau} (a+b)}$	Yes
Exponential distribution ${\displaystyle \operatorname {Exp} (\lambda )}$	${\displaystyle \operatorname {Exp} (\lambda )*\operatorname {Exp} (\lambda )=\operatorname {Erl} (\lambda ,2)=\Gamma (\lambda ,2)}$	No	yes, through ${\displaystyle \Gamma (\lambda ,1/n)}$
Erlang distribution ${\displaystyle \operatorname {Erl} (\lambda ,n)}$	${\displaystyle \operatorname {Erl} (\lambda ,n)*\operatorname {Erl} (\lambda ,m)=\operatorname {Erl} (\lambda ,n+m)}$	Yes on ${\displaystyle \mathbb {N} ^{+}}$	Yes, through ${\displaystyle \Gamma (\lambda ,m/n)}$
Gamma distribution ${\displaystyle \Gamma (p,r)}$	${\displaystyle \Gamma (p,r)*\Gamma (p,s)=\Gamma (p,r+s)}$	Yes on ${\displaystyle \mathbb {R} _{>0}}$	Yes, through ${\displaystyle \Gamma (p,r/n)}$
Chi-square distribution ${\displaystyle \chi ^{2}(m_{1})}$	${\displaystyle \chi ^{2}(m_{1})*\chi ^{2}(m_{2})=\chi ^{2}(m_{1}+m_{2})}$	Yes on ${\displaystyle \mathbb {N} ^{+}}$

literature

• Achim Klenke: Probability Theory . 3. Edition. Springer-Verlag, Berlin Heidelberg 2013, ISBN 978-3-642-36017-6 , doi : 10.1007 / 978-3-642-36018-3 .
• Hans-Otto Georgii: Stochastics . Introduction to probability theory and statistics. 4th edition. Walter de Gruyter, Berlin 2009, ISBN 978-3-11-021526-7 , doi : 10.1515 / 9783110215274 .
• Christian Hesse : Applied probability theory . 1st edition. Vieweg, Wiesbaden 2003, ISBN 3-528-03183-2 , doi : 10.1007 / 978-3-663-01244-3 .

Individual evidence

1. ^ Georgii: Stochastics. 2009, p. 75.