Absorbent state

Absorbing states are a term from the theory of Markov chains , which in turn are special stochastic processes in probability theory . Those states are called absorbent which can never be left again after entering.

definition

Given is a homogeneous Markov chain in discrete time with at most a countable state space . Then a subset is called absorbing, if and only if ${\displaystyle Z}$ ${\displaystyle A\subset Z}$

${\displaystyle P(X_{n+1}\in Z\setminus A\mid X_{n}\in A)=0}$

applies. Is one element, so ${\displaystyle A=\{k\}}$

${\displaystyle P(X_{n+1}\in Z\setminus \{k\}\mid X_{n}=k)=0}$,

then the state is called the absorbing state. Continue to be called ${\displaystyle k}$

${\displaystyle \tau _{A}^{i}:=\min\{n\in \mathbb {N} :X_{n}\in A{\text{ und }}X_{0}=i\}}$

Time of first entry into the absorbing quantity and

${\displaystyle a_{i}:=P(\tau _{A}^{i}<\infty )}$

the probability of absorption.

Calculation of the absorption probabilities

Let a homogeneous Markov chain with state space and several absorbing sets be given. To calculate the absorption probabilities in an absorbing quantity , the following approach is suitable : combine all absorbing quantities except in the quantity and consider for all ${\displaystyle Z}$ ${\displaystyle A}$ ${\displaystyle A}$ ${\displaystyle B}$ ${\displaystyle i\in Z\setminus (A\cup B)}$

${\displaystyle {\begin{aligned}P(\tau _{A}^{i}<N+1)&=\sum _{j\in Z}p_{ij}P(\tau _{A}^{j}<N)\\\Rightarrow a_{i}&=\sum _{j\in Z}p_{ij}a_{j}{\text{ für }}\ N\to \infty .\end{aligned}}}$

In the case of a finite state space, this corresponds exactly to a solution of the right eigenvector problem for the eigenvalue 1 of the transition matrix . In general, the solution is not clear. This is achieved by adding the boundary conditions if (when starting in the absorbing state, absorption has already occurred) and if (when starting in a different absorbing state, this can never be achieved). ${\displaystyle a_{i}=1}$ ${\displaystyle i\in A}$ ${\displaystyle a_{i}=0}$ ${\displaystyle i\in B}$ ${\displaystyle A}$

If one now wants to know in which state the absorption occurs, considerations analogous to the above lead to the additional secondary condition if and if . ${\displaystyle k\in A}$ ${\displaystyle a_{i}=1}$ ${\displaystyle i=k}$ ${\displaystyle a_{i}=0}$ ${\displaystyle \ i\in A\setminus \{k\}}$

Examples

Consider the following transition matrix, which represents a homogeneous Markov chain on the state space. There are two absorbent quantities: the condition and the quantity . State 1 is only reached by state 2 with probability 0.1, states 4 and 5 only by states 2 (with probability 0.1) or 3 (with probability 0.5). ${\displaystyle Z=\{1,2,3,4,5\}}$ ${\displaystyle \{1\}=A}$ ${\displaystyle B=\{4,5\}}$

${\displaystyle P={\begin{bmatrix}1&0&0&0&0\\0{,}1&0&0{,}8&0{,}1&0\\0&0{,}5&0&0&0{,}5\\0&0&0&0&1\\0&0&0&1&0\end{bmatrix}}}$

The eigenspace is then We are interested in the absorption probability in state 1. Therefore we are now looking for a vector that satisfies and . The second vector in the span fulfills the second condition, the first is obtained by normalizing the vector. This means that the vector is the i-th component that contains the absorption probability when starting in i in state 1. If one now wants to calculate the absorption probability in , one must demand and . It then follows . It is nice to see here that the absorption probabilities add up to one, that means no matter where you start, you are always absorbed either in state 1 or in states 4 and 5. ${\displaystyle \operatorname {span} ((-11,-1,0,1,1)^{T};(12,2,1,0,0)^{T})}$ ${\displaystyle a_{1}=1}$ ${\displaystyle a_{4}=a_{5}=0}$ ${\displaystyle a=(a_{i})_{i=1\cdots 5}=(1,1/6,1/12,0,0)^{T}}$ ${\displaystyle A={4,5}}$ ${\displaystyle a_{1}=0}$ ${\displaystyle a_{4}=a_{5}=1}$ ${\displaystyle a=(a_{i})_{i=1\cdots 5}=(0,5/6,11/12,1,1)}$

If one is interested in the absorption probability in state 4, the direct application of the system of equations and the secondary conditions yields and ${\displaystyle a_{1}=a_{5}=0,a_{4}=1}$

${\displaystyle {\begin{aligned}-a_{2}&+0{,}8a_{3}&=-0{,}1\\0{,}5a_{2}&-a_{3}&=0.\end{aligned}}}$

This gives the solution . ${\displaystyle a=(a_{i})_{i=1\cdots 5}=(0,1/6,1/12,1,0)}$

use

Absorbent states are particularly interesting in population dynamics and in the modeling of games. In population dynamics, the time steps of a Markov chain can be understood as generations, the state space is then the living individuals. Here an absorbing state would be the 0, i.e. the fact that the species has become extinct. In this case, the probability of extinction and the probability of absorption are equivalent in the 0. A typical example here is the Galton-Watson trial . Some games of chance can also be studied well with the help of Markov chains. The time of the game round and the state space correspond to the money of a player. An absorbing state would be 0 (bankruptcy of the player) or the total stake (bankruptcy of the other player). So the absorption probability is the profit / loss probability.

literature

• Ulrich Krengel : Introduction to probability theory and statistics . 8th edition, Vieweg, 2005. ISBN 978-3-834-80063-3
• Hans-Otto Georgii: Stochastics: Introduction to Probability Theory and Statistics , 4th edition, de Gruyter, 2009. ISBN 978-3-110-21526-7
• Christian Hesse: Applied probability theory : a well-founded introduction with over 500 realistic examples and tasks, Vieweg, Braunschweig / Wiesbaden 2003, ISBN 978-3-528-03183-1 .