Convergence (stochastics)

In stochastics there are different concepts of a limit value concept for random variables . Unlike in the case of real number sequences, there is no natural definition for the limiting behavior of random variables with increasing sample size , because the asymptotic behavior of the experiments always depends on the individual realizations and we are therefore formally dealing with the convergence of functions . Therefore, concepts of different strengths have emerged over the course of time; the most important of these types of convergence are briefly presented below.

requirements

We will always formulate the classical concepts of convergence in the following model: A sequence of random variables is given, which are defined on a probability space and map into the same normalized space . With its Borel algebra, this image space naturally becomes a measurement space . In order to understand the core statements, it is sufficient to always imagine real random variables . On the other hand, the following definitions can be generalized in an obvious manner to the case of metric spaces as image space. ${\displaystyle (X_{n})_{(n\in \mathbb {N} )}\;}$ ${\displaystyle (\Omega ,\Sigma ,P)}$

A realization of this sequence is usually referred to as. ${\displaystyle X_{n}(\omega )}$

Almost certain convergence

→ Main article : Almost certain convergence

The concept of almost certain convergence can best be compared with the formulation for sequences of numbers. It is mainly used in the formulation of strong laws of large numbers .

The sequence is said to almost certainly converge to a random variable if ${\displaystyle X_{n}}$ ${\displaystyle X}$

${\displaystyle P\left(\lim _{n\to \infty }X_{n}=X\right)=P\left(\left\{\omega \in \Omega \,\left|\,\lim _{n\to \infty }X_{n}(\omega )=X(\omega )\right.\right\}\right)=1}$

applies and then writes . Translated, this means that the classic concept of convergence with regard to the norm applies to almost all implementations of the sequence. The almost certain convergence thus corresponds to the point-by-point convergence μ-almost everywhere from measurement theory . ${\displaystyle X_{n}{\xrightarrow {\text{ f. s. }}}X}$

Convergence in the p th mean

→ Main article : Convergence in the p-th mean

An integration theoretical approach is pursued with the concept of convergence in the -th mean ${\displaystyle p}$ . Individual realizations are not considered here, but rather expected values ​​of the random variables.

Formally, the -th mean converges to a random variable , if ${\displaystyle X_{n}\;}$ ${\displaystyle p}$ ${\displaystyle X}$

${\displaystyle \lim _{n\rightarrow \infty }E[|X_{n}-X|^{p}]=0}$

applies. It is assumed. This means that the difference in the L ^p -space against converges. This convergence is therefore also called convergence. ${\displaystyle p\geq 1\;}$ ${\displaystyle X_{n}-X\;}$ ${\displaystyle {\mathcal {L}}^{p}(P)}$ ${\displaystyle 0}$ ${\displaystyle {\mathcal {L}}^{p}}$

Because of the inequality of the generalized mean values, the convergence in the -th mean follows from the convergence in the -th mean. ${\displaystyle q>p\;}$ ${\displaystyle q}$ ${\displaystyle p}$

Convergence in probability

→ Main article : Convergence in probability

A somewhat weaker concept of convergence is stochastic convergence or convergence in probability . As the name suggests, special realizations of the random variables are not considered, but probabilities for certain events. A classic application of stochastic convergence are weak laws of large numbers .

The mathematical formulation is: The sequence converges stochastically to a random variable , if ${\displaystyle X_{n}\;}$ ${\displaystyle X}$

${\displaystyle \forall \varepsilon >0\colon \lim _{n\to \infty }P(|X_{n}-X|>\varepsilon )=0}$.

The following notations are usually used for convergence in probability: or or . ${\displaystyle \operatorname {P-lim} _{n\rightarrow \infty }\,X_{n}=X\;}$ ${\displaystyle \operatorname {plim} (X_{n})=X}$ ${\displaystyle X_{n}{\stackrel {P}{\rightarrow }}X}$

The stochastic convergence corresponds to the convergence according to the measure from the measure theory.

Weak convergence

→ Main article : Convergence in distribution

The fourth prominent concept of convergence is that of convergence in distribution , sometimes also called weak convergence (for random variables). It corresponds to the weak convergence for measures in measure theory.

A sequence of random variables converges in distribution to the random variable if the sequence of the induced image measures converges weakly towards the image measure. That is, for all continuous bounded functions applies ${\displaystyle X_{n}\;}$ ${\displaystyle X}$ ${\displaystyle \mu _{n}(A):=P(X_{n}\in A)}$ ${\displaystyle \mu (A):=P(X\in A)}$ ${\displaystyle f}$

${\displaystyle \lim _{n\to \infty }E(f\circ X_{n})=E(f\circ X)}$.

For real random variable according to the set of Helly-Bray , the following characterization equivalently: For the distribution functions of and of true ${\displaystyle F_{n}}$ ${\displaystyle X_{n}}$ ${\displaystyle F}$ ${\displaystyle X}$

${\displaystyle \lim _{n\to \infty }F_{n}(x)=F(x)}$

in all places where is steady. The best known applications of convergence in distribution are central limit theorems . ${\displaystyle x\in \mathbb {R} }$ ${\displaystyle F}$

Since the convergence in distribution is defined exclusively by the image dimensions or by the distribution function of the random variables, it is not necessary for the random variables to be defined in the same probability space.

The notation used is usually or , but sometimes also . The letters “W” and “D” stand for the corresponding terms in English, i.e. weak convergence or convergence in distribution . ${\displaystyle X_{n}{\stackrel {w}{\rightarrow }}X}$ ${\displaystyle X_{n}{\stackrel {\mathcal {D}}{\rightarrow }}X}$ ${\displaystyle X_{n}\implies X}$

Relationship between the individual types of convergence

In the series of the most important convergence terms in stochastics, the two terms presented first represent the strongest types of convergence. The stochastic convergence of a sequence of random variables can always be derived from both the almost certain convergence and the convergence in the p-th mean . Furthermore, the convergence in distribution , which is the weakest of the convergence types presented here, automatically follows from stochastic convergence . So compact applies

${\displaystyle {\begin{matrix}{\text{Fast sichere}}\\{\text{Konvergenz}}\end{matrix}}\implies {\begin{matrix}{\text{Konvergenz in}}\\{\text{Wahrscheinlichkeit}}\end{matrix}}\implies {\begin{matrix}{\text{Konvergenz in}}\\{\text{Verteilung}}\end{matrix}}}$

and

${\displaystyle {\begin{matrix}{\text{Konvergenz im}}\\{\text{p-ten Mittel}}\end{matrix}}\implies {\begin{matrix}{\text{Konvergenz in}}\\{\text{Wahrscheinlichkeit}}\end{matrix}}\implies {\begin{matrix}{\text{Konvergenz in}}\\{\text{Verteilung}}\end{matrix}}}$.

In exceptional cases, other implications also apply: If a sequence of random variables in distribution converges against a random variable X and X is almost certainly constant, then this sequence also converges stochastically.

The convergence in the p-th mean does not generally result in an almost certain convergence. Conversely, from almost certain convergence, no convergence in the p-th mean can generally be concluded. However, this conclusion is permissible if there is a common majorante in (see theorem of majorized convergence ). A sequence of random variables converges if and only if it converges stochastically and is equally integrable . ${\displaystyle L^{p}}$ ${\displaystyle L^{1}}$

example

On the probability space with , the Borel sets and the Borel-Lebesgue measure , consider the random variable and the sequence of random variables, which is defined as follows for with (every natural has a unique decomposition of this kind): ${\displaystyle (\Omega ,\Sigma ,P)}$ ${\displaystyle \Omega =[0,1]}$ ${\displaystyle \Sigma }$ ${\displaystyle P}$ ${\displaystyle X(\omega )=0}$ ${\displaystyle X_{n}(\omega )}$ ${\displaystyle n=2^{k}+m}$ ${\displaystyle 0\leq m<2^{k}}$ ${\displaystyle n}$

${\displaystyle X_{n}(\omega )={\begin{cases}1&{\text{falls }}{\frac {m}{2^{k}}}\leq \omega \leq {\frac {m+1}{2^{k}}}\\0&{\text{sonst.}}\end{cases}}}$

The functions are, so to speak, increasingly thinner spikes that run over the interval . ${\displaystyle X_{n}}$ ${\displaystyle [0,1]}$

Because of

${\displaystyle E[|X_{n}-X|^{p}]=\int _{0}^{1}|X_{n}(\omega )-0|^{p}d\omega ={\frac {1}{2^{k}}}\to 0}$

converges in the p-th mean to . From the above-described relationship between the individual types of convergence, it follows that converges stochastically against as well as from ${\displaystyle X_{n}}$ ${\displaystyle X}$ ${\displaystyle X_{n}}$ ${\displaystyle X}$

${\displaystyle P(|X_{n}-X|>\varepsilon )={\begin{cases}{\frac {1}{2^{k}}}\;&{\text{für}}\;0<\varepsilon \leq 1\\0\;&{\text{für}}\;\varepsilon >1\end{cases}}}$

and because of for , so ${\displaystyle k\rightarrow \infty }$ ${\displaystyle n\rightarrow \infty }$

${\displaystyle P(|X_{n}-X|>\varepsilon )\leq {\frac {1}{2^{k}}}\to 0\;{\text{für jedes}}\;\varepsilon >0}$

reveals.

For every fixed one , however, applies to infinitely many , the same applies to infinitely many , so that there is no almost certain convergence of . For each subsequence of , however, a subsequence can be found that almost certainly converges against . If there were a topology of almost certain convergence, it would follow from this property that it almost certainly converges against . This example also shows that there can be no topology of almost certain convergence. ${\displaystyle \omega \in [0,1]}$ ${\displaystyle X_{n}(\omega )=1}$ ${\displaystyle n}$ ${\displaystyle X_{n}(\omega )=0}$ ${\displaystyle n}$ ${\displaystyle X_{n}}$ ${\displaystyle X_{n_{i}}}$ ${\displaystyle X_{n}}$ ${\displaystyle X_{n_{i_{j}}}}$ ${\displaystyle X}$ ${\displaystyle X_{n}}$ ${\displaystyle X}$

See also

• Dimension-theoretical convergence terms

literature

• Heinz Bauer : Probability Theory . 4th edition. De Gruyter, Berlin 1991, ISBN 3-11-012190-5 , pp. 34 (convergence of random variables and distributions).
• Heinz Bauer: Measure and integration theory . 2nd Edition. De Gruyter, Berlin 1992, ISBN 3-11-013625-2 , §15 Convergence Theorems and §20 Stochastic Convergence, p. 91 ff. and 128 ff .
• Jürgen Elstrodt : Measure and integration theory . 7th edition. Springer, Berlin 2011, ISBN 978-3-642-17904-4 , Chapter VI. Convergence terms in measure and integration theory, p. 219–268 (describes in detail the relationships between the various types of convergence).
• Christian Hesse : Applied probability theory . 1st edition. Vieweg, Wiesbaden 2003, ISBN 3-528-03183-2 , pp. 216-238 , doi : 10.1007 / 978-3-663-01244-3 .

Individual evidence

1. ^ Robert B. Ash: Real Analysis and Probability . Academic Press, New York 1972, ISBN 0-12-065201-3 , Theorem 4.5.4.
2. ^ Robert B. Ash: Real Analysis and Probability . Academic Press, New York 1972, ISBN 0-12-065201-3 , Theorem 2.5.5.
3. ^ Robert B. Ash: Real Analysis and Probability . Academic Press, New York 1972, ISBN 0-12-065201-3 , Theorem 2.5.1.
4. ^ Virtual Laboratories in Probability and Statistics, Excercise 2.8.3
5. ^ Robert B. Ash: Real Analysis and Probability . Academic Press, New York 1972, ISBN 0-12-065201-3 , Examples 2.5.6.
6. ↑ Bernard R. Gelbaum, John MH Olmsted: counterexamples in Analysis . Dover Publications, Mineola, New York 2003, ISBN 0-486-42875-3 , Section 8.40, Sequences of functions converging in different senses, pp. 109-111 .
7. ↑ J. Cigler, H.-C. Reichel: topology. A basic lecture . 6th edition. Bibliographisches Institut, Mannheim 1978, ISBN 3-411-00121-6 , p. 88 .