Shannon (unit)

Shannon (abbreviated Sh ) is the unit for the information content of a message named after the American mathematician and founder of information theory, Claude Elwood Shannon .

definition

A Shannon (1 Sh) is a unit of the amount of information in a message. The message is described by a set of characters and by the probabilities p _i with which the individual characters appear in the message. By definition, such a message has the following informational content, measured in Shannon, of ${\displaystyle Z=\{z_{1},z_{2},\dots ,z_{S}\}}$

${\displaystyle \mathrm {H} =-\sum _{i}{p_{i}\cdot \log _{2}p_{i}}\quad [{\text{Sh}}]}$

The unit Sh results from the information theory established by Claude Shannon and corresponds to the entropy of the relevant news source and, in biometry, to the Shannon index of a population of different species.

Examples

• The simplest example: A one-off message from the character set , where both characters occur with a probability of 0.5 each, has the information content . ${\displaystyle Z=\{0,1\}}$ ${\displaystyle -{\frac {1}{2}}\;\log _{2}\!\left({\frac {1}{2}}\right)-{\frac {1}{2}}\;\log _{2}\!\left({\frac {1}{2}}\right)=-\log _{2}\!\left({\frac {1}{2}}\right)=\log _{2}(2)=1\,{\text{Sh}}}$

• If the character set consists of ten characters, for example , with all characters occurring again with the same probability, then the information content is . This can be interpreted in such a way that a 4-bit coding is definitely sufficient to transmit such messages (this is implemented, for example, by the BCD code ), but that there is still redundancy . With the BCD code, this redundancy manifests itself in such a way that only 10 of the 16 different characters that can be represented with this code actually occur. ${\displaystyle \{0,1,2,3,4,5,6,7,8,9\}}$ ${\displaystyle -\log _{2}\left({\frac {1}{10}}\right)=\log _{2}(10)\approx 3{,}322\ \mathrm {Sh} }$

standardization

Definitions of the units of decision content in information theory can be found in the international standards IEC 60027-3 and ISO 2382-16. In addition to the Shannon unit based on the logarithm for base 2, the Hartley based on the logarithm of ten (unit symbol: Hart) and the natural information unit based on the natural logarithm (unit symbol: nat) are explained.

"The attempt by the International Organization for Standardization (ISO) to replace the unit bit with the Shannon unit in 1975 [...] was not very successful in retrospect."

- Walter Umstätter

literature

• Johann Blieberger, Bernd Burgstaller, Gerhard Helge Schildt: Computer Science - Basics . 4th edition, Springer Verlag, Vienna 2002, ISBN 978-3-211-83710-8 .

Web links

• Roland Brackmann: Shannon's theory . April 28, 2005, accessed on February 26, 2018 (PDF; 160 kB).
• Information, entropy, Shannon's coding theorem . April 21, 2008, accessed on February 26, 2018 (PDF; 243 kB).
• RG Spallek: Computer architecture I - information display . 2004, accessed on February 26, 2018 (PDF; 117 kB).

Individual evidence

1. ^ News for Documentation 49 (4), 1998, pp. 221–224.