Entropy rate

The entropy rate ( English entropy rate ) enables the information theory independent of the length of a message is a measurement of the entropy based on a character.

Formally, it can be defined as follows:

${\displaystyle H_{\mathrm {Rate} }={\frac {1}{n}}H(X_{1},...,X_{n})}$ ${\displaystyle =-{1 \over n}\sum _{x_{1},...,x_{n}}p(x_{1},...,x_{n})\log p(x_{1},...,x_{n})}$.

Here is the number of characters in the message or the number of random variables . ${\displaystyle n}$

The entropy rate enables the entropies of messages of different lengths to be compared.

literature

• Peter Adam Höher: Basics of digital information transfer . From theory to mobile radio applications, 2nd edition, Springer Vieweg, Wiesbaden 2013, ISBN 978-3-8348-1784-6 .

Web links

• Information Theory - (P) Review (accessed February 12, 2018)
• Stochastic Processes (accessed February 12, 2018)
• Universal encodings and Lempel-Ziv encoding (accessed on February 12, 2018)
• Information and communication (accessed on February 12, 2018)