Independently and identically distributed random variables

Independently and identically distributed random variables are a central construction of stochastics and an important prerequisite for many mathematical theorems of statistics . Independently and identically distributed random variables all have the same distribution , i.e. they assume the same values ​​with the same probability, but do not influence each other. Thus, independently and identically distributed random variables are the stochastic modeling of a general scientific experiment: the independence ensures that the individual experiments do not influence each other, the identical distribution ensures that the same experiment is carried out over and over again.

The abbreviations in the literature include iid or iid as an abbreviation of the English independent and identically distributed or the German-based uiv

definition

A sequence of random variables is given . This is called independent and identically distributed if the following two criteria are met: ${\displaystyle (X_{n})_{n\in \mathbb {N} }}$

• The family of random variables in the sequence are stochastically independent random variables . ${\displaystyle X_{n}}$
• The random variables all have the same distribution. That means there is a probability distribution so that for everyone . ${\displaystyle P}$ ${\displaystyle X_{n}\sim P}$ ${\displaystyle n\in \mathbb {N} }$

variants

The definition can easily be extended to any number of index sets. The definition for the index set , i.e. for a finite sequence of random variables , or for any, possibly uncountable index set is common . ${\displaystyle \{0,1,\dots ,N\}}$ ${\displaystyle (X_{n})_{n\in \{0,1,\dots ,N\}}}$ ${\displaystyle I}$

existence

A fundamental question is whether there are sequences of infinitely many random variables that are independently and identically distributed. This is not obvious since stochastic independence is a strong property that is defined on the underlying set system and it is not clear a priori whether a set system exists that is large enough to allow the stochastic independence of many random variables.

In fact, with advanced methods it can be shown that arbitrarily large families of independently and identically distributed random variables exist. In introductory literature, a simplified construction is usually given and the statement that provides the existence of independently and identically distributed random variables is also referred to as the "clone sentence".

In essence, many constructions are based on the existence of the infinite product dimension , guaranteed in the most general version by the Andersen-Jessen theorem . The product space is used to construct a sequence of independent, identically distributed random variables with distributions on the real numbers ${\displaystyle P}$

${\displaystyle (\Omega ,{\mathcal {A}},Q)=(\mathbb {R} ^{\mathbb {N} },\left({\mathcal {B}}(\mathbb {R} )\right)^{\otimes \mathbb {N} },P^{\mathbb {N} })}$

constructed, where is the product σ-algebra and the infinite product measure. ${\displaystyle \left({\mathcal {B}}(\mathbb {R} )\right)^{\otimes \mathbb {N} }}$ ${\displaystyle P^{\mathbb {N} }}$

Then you define

${\displaystyle X_{n}\colon (\Omega ,{\mathcal {A}},Q)\to (\mathbb {R} ,{\mathcal {B}}(\mathbb {R} ))}$

as a projection of the nth component, all random variables (since they are measurable due to the definition of the product σ-algebra) are identically distributed and stochastically independent. The more general constructions with more general image spaces and index sets run analogously. ${\displaystyle (X_{n})_{n\in \mathbb {N} }}$

Related concepts

Interchangeable family of random variables

→ Main article : Interchangeable family of random variables

An interchangeable family of random variables is a family of random variables in which the distribution of the entire family does not change if a finite number of the random variables are interchanged. Interchangeable families are always distributed identically; conversely, every independently and identically distributed family is always interchangeable.

Conditionally independent and identically distributed

An analogous term to independently identically distributed random variables is obtained by replacing the independence of set systems , on which the independence of random variables is based, with the conditional independence (of set systems) based on the conditional expected value . Then a family of random variables is called independently and identically distributed , if ${\displaystyle (X_{i})_{i\in I}}$ ${\displaystyle {\mathcal {A}}}$

• the σ-algebras generated by the random variables are conditionally given independently and ${\displaystyle \sigma (X_{i})}$ ${\displaystyle {\mathcal {A}}}$
• the conditional distributions are all the same. ${\displaystyle P(\{X_{i}\in \cdot \}|{\mathcal {A}})}$

More related terms

In particular in the area of ​​the classic limit theorems of stochastics ( law of large numbers and central limit theorem ) there are various modifications of the requirement that a sequence should be distributed independently and identically. Here, for example, the stochastic independence is replaced by

• pairwise stochastic independence, i.e. the stochastic independence from and for . This is a real weakening compared to the stochastic independence of the entire family from random variables, as an example of the stochastic independence of events here demonstrates . ${\displaystyle X_{i}}$ ${\displaystyle X_{j}}$ ${\displaystyle i\neq j}$
• Uncorrelatedness , whereby this is always only defined as pairwise uncorrelatedness. Since independence always results in uncorrelatedness, but the reverse is generally not true, this is a real weakening compared to paired independence (and thus also independence).

Another modification are the independent schemes of random variables , such as those found in Lindeberg-Feller's central limit value theorem . The random variables of a sequence are grouped and stochastic independence is only required within the groups. The dependencies between the groups are irrelevant.

literature

• Achim Klenke: Probability Theory . 3. Edition. Springer-Verlag, Berlin Heidelberg 2013, ISBN 978-3-642-36017-6 , doi : 10.1007 / 978-3-642-36018-3 .
• Claudia Czado, Thorsten Schmidt: Mathematical Statistics . Springer-Verlag, Berlin Heidelberg 2011, ISBN 978-3-642-17260-1 , doi : 10.1007 / 978-3-642-17261-8 .
• Norbert Kusolitsch: Measure and probability theory . An introduction. 2nd, revised and expanded edition. Springer-Verlag, Berlin Heidelberg 2014, ISBN 978-3-642-45386-1 , doi : 10.1007 / 978-3-642-45387-8 .
• Klaus D. Schmidt: Measure and Probability . 2nd, revised edition. Springer-Verlag, Heidelberg Dordrecht London New York 2011, ISBN 978-3-642-21025-9 , doi : 10.1007 / 978-3-642-21026-6 .
• Ehrhard Behrends: Elementary Stochastics . A learning book - co-developed by students. Springer Spectrum, Wiesbaden 2013, ISBN 978-3-8348-1939-0 , doi : 10.1007 / 978-3-8348-2331-1 .

Individual evidence

1. ↑ Klenke: Probability Theory. 2013, p. 57.
2. ^ Schmidt: Measure and probability. 2011, p. 347.
3. ↑ Kusolitsch: Measure and probability theory. 2014, p. 246.
4. ↑ Czado, Schmidt: Mathematical Statistics. 2011, p. 7.
5. ↑ Klenke: Probability Theory. 2013, p. 57.
6. ↑ Behrends: Elementary Stochastics. 2013, p. 141.
7. ↑ Klenke: Probability Theory. 2013, p. 243.