Kolmogorov-Sinai entropy

The Kolmogorow-Sinai entropy is an invariant of dimension-preserving maps in the mathematical sub-area of dynamic systems . It generalizes the entropy concept known from probability theory (and originally from thermodynamics ) . The entropy is supposed to measure how much information one receives with each new step of the dynamic system. Its definition goes back to Andrei Kolmogorow , but it was not until the late 1950s that Jakow Sinai succeeded in demonstrating the non-triviality of this invariant. For this, among other things, Sinai received the Abel Prize in 2014 .

The Kolmogorov-Sinai entropy intuitively measures the chaotic nature of dynamic systems. It is trivial, i. that is, its value is zero for non-chaotic mappings such as rotations.

It is also referred to as mass-theoretical entropy or metric entropy or, for short, KS entropy .

definition

Let it be a probability space and a measure-preserving map . ${\ displaystyle (X, \ Sigma, P)}$ ${\ displaystyle f \ colon X \ to X}$

As is well known, in probability theory the entropy of a partition (i.e. a disjoint decomposition ) is defined by ${\ displaystyle Q}$ ${\ displaystyle X = Q_ {1} \ cup \ ldots \ cup Q_ {r}}$

{\ displaystyle H (Q): = - \ sum _ {i = 1} ^ {r} P (Q_ {i}) \ log P (Q_ {i})}

.

The entropy of with respect to the partition is then defined as ${\ displaystyle f}$ ${\ displaystyle Q}$

{\ displaystyle h (f, Q): = \ lim _ {N \ rightarrow \ infty} {\ frac {1} {N}} H \ left (\ bigvee _ {n = 0} ^ {N} f ^ { -n} Q \ right)}

,

in which

{\ displaystyle \ bigvee _ {n = 0} ^ {N} f ^ {- n} Q = \ left \ {Q_ {i_ {0}} \ cap f ^ {- 1} Q_ {i_ {1}} \ cap \ cdots \ cap f ^ {- N} Q_ {i_ {N}} {\ text {with}} i _ {\ ell} = 1, \ ldots, k, \ \ ell = 0, \ ldots, N, \ P \ left (Q_ {i_ {0}} \ cap f ^ {- 1} Q_ {i_ {1}} \ cap \ cdots \ cap f ^ {- N} Q_ {i_ {N}} \ right)> 0 \ right \}}

.

(The existence of the limit value is the statement of the Shannon-MacMillan Theorem .)

Finally, the Kolmogorov-Sinai entropy of the measure-preserving mapping is defined as the supremum of over all partitions : ${\ displaystyle f}$ ${\ displaystyle h (f, Q)}$ ${\ displaystyle Q}$

{\ displaystyle h (f) = \ sup _ {Q} h (f, Q).}

Generating partitions (Sinai theorem)

The Sinai theorem says that one can effectively compute the Kolmogorov-Sinai entropy by means of generating partitions.

Definition : A generating partition of a measure-maintaining dynamic system is a finite decomposition , so that is the smallest σ-algebra that contains all ( ). ${\ displaystyle (X, \ Sigma, P, f)}$ ${\ displaystyle X = Q_ {1} \ cup \ ldots \ cup Q_ {r}}$ ${\ displaystyle \ Sigma}$ ${\ displaystyle f ^ {n} Q_ {i}}$ ${\ displaystyle n \ in \ mathbb {Z}, i \ in \ left \ {1, \ ldots, r \ right \}}$

Theorem (Sinai) : If is the generating partition of a measure-maintaining dynamic system , then is ${\ displaystyle Q}$ ${\ displaystyle (X, \ Sigma, P, f)}$

{\ displaystyle h (f) = h (f, Q)}

.

Examples

For a rotation of the circle (or more generally the n-dimensional torus ) the entropy is trivial:

{\ displaystyle h (f) = 0}

.

For the self-mapping of the n-dimensional torus defined by an integer unimodular matrix is ${\ displaystyle A \ in GL (n, \ mathbb {Z})}$ ${\ displaystyle f \ colon T ^ {n} \ to T ^ {n}}$ ${\ displaystyle T ^ {n} = \ mathbb {R} ^ {n} / \ mathbb {Z} ^ {n}}$

{\ displaystyle h (f) = \ sum _ {\ mid \ lambda _ {i} \ mid> 1} \ log (\ mid \ lambda _ {i} \ mid)}

,

where the eigenvalues of (possibly counted according to their multiplicity ). This was proven by Sinai in 1959 and was the first example of a mapping of nontrivial entropy.

{\ displaystyle \ lambda _ {1}, \ ldots, \ lambda _ {n}}

{\ displaystyle A}

literature

P. Billingsley: Ergodic Theory and Information. J. Wiley, 1965.
IP Cornfeld, SF Fomin, Ya.G. Sinai: Ergodic Theory. Springer, 1981.
AN Kolmogorov: New Metric Invariant of Transitive Dynamical Systems and Endomorphisms of Lebesgue Spaces. In: Doklady of Russian Academy of Sciences. 119, No. 5, 1958 pp. 861-864.
AN Kolmogorov: Entropy per unit time as a metric invariant of automorphism. In: Doklady of Russian Academy of Sciences. 124, 1959, pp. 754-755.
E. Lindenstrauss, Y. Peres, W. Schlag: Bernoulli convolutions and intermediate values for entropy of K-partitions. In: J. Anal. Math. 87, 2002, pp. 337-367.
W. Parry: Entropy and Generators in Ergodic Theory. WA Benjamin, Inc., New York / Amsterdam 1969.
Ya.G. Sinai: On the Notion of Entropy of a Dynamical System. In: Doklady of Russian Academy of Sciences. 124, 1959, pp. 768-771.
P. Walters: An Introduction to Ergodic Theory. Springer, New York / Berlin, 1969-

Web links

The entropy of a dynamical system . (Popular scientific presentation on the occasion of the Abel Prize for Sinai)
Sinai: Kolmogorov-Sinai entropy . Scholarpedia.
Austin: Entropy and Sinai's Theorem .

Individual evidence

^ Katok-Hasselblatt: Introduction to the modern theory of dynamical systems. (= Encyclopedia of Mathematics and its Applications. 54). Cambridge University Press, Cambridge, 1995, ISBN 0-521-34187-6 , section 4.4
^ Sinai: On the concept of entropy for a dynamic system. In: Dokl. Akad. Nauk SSSR. 124, 1959 (Russian).

[1] Katok-Hasselblatt: Introduction to the modern theory of dynamical systems. (= Encyclopedia of Mathematics and its Applications. 54). Cambridge University Press, Cambridge, 1995, ISBN 0-521-34187-6 , section 4.4

[2] Sinai: On the concept of entropy for a dynamic system. In: Dokl. Akad. Nauk SSSR. 124, 1959 (Russian).