Infomax

Infomax is an optimization principle of artificial neural networks and other information processing systems. It states that a function that a number of input values I a number of output values O attributes should be chosen or learned that the average mutual information according to Shannon between I and O is maximized. This takes place depending on predefined conditions and / or existing noise in the signal. Infomax algorithms are learning algorithms that serve this optimization process.

Infomax refers to the principle of redundancy reduction , which was formulated by Horace Barlow for the description of biological stimulus processing in 1961. Atick and Redlich used it to calculate processing processes in the retina .

One of the main uses of Infomax is in independence analysis (ICA), where independent signals are found by maximizing entropy .

literature

Bell AJ, Sejnowski TJ: The "Independent Components" of Natural Scenes are Edge Filters . In: Vision Res. . 37, No. 23, December 1997, pp. 3327-38. doi : 10.1016 / S0042-6989 (97) 00121-1 . PMID 9425547 . PMC 2882863 (free full text).
Linsker R: A local learning rule that enables information maximization for arbitrary input distributions . In: Neural Computation . 9, No. 8, 1997, pp. 1661-65. doi : 10.1162 / neco.1997.9.8.1661 .
Stone, James V., Dr .: Independent component analysis: a tutorial introduction . MIT Press, Cambridge, Mass. 2004, ISBN 0-262-69315-1 .

Individual evidence

↑ Linsker R: Self-organization in a perceptual network Archived from the original on December 5, 2014. Info: The archive link was automatically inserted and not yet checked. Please check the original and archive link according to the instructions and then remove this notice. In: IEEE Computer . 21, No. 3, 1988, pp. 105-17. doi : 10.1109 / 2.36 . @1@ 2

↑ Barlow, H .: Possible principles underlying the transformations of sensory messages . In: Rosenblith, W. (Ed.): Sensory Communication . MIT Press, Cambridge MA 1961, pp. 217-234.

↑ Atick JJ, Redlich AN: What does the retina know about natural scenes? . In: Neural Computation . 4, No. 2, 1992, pp. 196-210. doi : 10.1162 / neco.1992.4.2.196 .

↑ Bell AJ, Sejnowski TJ: An information-maximization approach to blind separation and blind deconvolution . In: Neural Comput . 7, No. 6, November 1995, pp. 1129-59. doi : 10.1162 / neco.1995.7.6.1129 . PMID 7584893 .

↑ Nadal JP, Parga N .: Sensory coding: information maximization and redundancy reduction . In: Neural information processing, G. Burdet, P. Combe and O. Parodi Eds., World Scientific Series in Mathematical Biology and Medecine . 7, 1999, pp. 164-171.

[1] Linsker R: Self-organization in a perceptual network Archived from the original on December 5, 2014. Info: The archive link was automatically inserted and not yet checked. Please check the original and archive link according to the instructions and then remove this notice. In: IEEE Computer . 21, No. 3, 1988, pp. 105-17. doi : 10.1109 / 2.36 . @1@ 2

[2] Barlow, H .: Possible principles underlying the transformations of sensory messages . In: Rosenblith, W. (Ed.): Sensory Communication . MIT Press, Cambridge MA 1961, pp. 217-234.

[3] Atick JJ, Redlich AN: What does the retina know about natural scenes? . In: Neural Computation . 4, No. 2, 1992, pp. 196-210. doi : 10.1162 / neco.1992.4.2.196 .

[4] Bell AJ, Sejnowski TJ: An information-maximization approach to blind separation and blind deconvolution . In: Neural Comput . 7, No. 6, November 1995, pp. 1129-59. doi : 10.1162 / neco.1995.7.6.1129 . PMID 7584893 .

[5] Nadal JP, Parga N .: Sensory coding: information maximization and redundancy reduction . In: Neural information processing, G. Burdet, P. Combe and O. Parodi Eds., World Scientific Series in Mathematical Biology and Medecine . 7, 1999, pp. 164-171.