Maximum entropy method

from Wikipedia, the free encyclopedia

The maximum entropy method or MEM is a method of Bayesian statistics that allows an a priori probability to be assigned despite inadequate problem-specific information . It replaces earlier approaches such as the “ principle of insufficient reason ” formulated by Laplace .

Origin and procedure

The method was introduced in 1957 by Edwin Thompson Jaynes based on methods of statistical mechanics and Shannon's information theory. The basis is to maximize the entropy of the a priori probability in the absence of information , since any other assignment would impose arbitrary restrictions on the situation under consideration. The maximum entropy method is fixed as little as possible. According to Jaynes, however, this is only the last step in filling in all the information that may still exist.

In statistical physics this means: “ Among all the states of a physical system that are compatible with the existing knowledge about the system, one has to choose that which maximizes the entropy. "

The method is used for the optimal extraction of information from noisy signals depending on the signal-to-noise ratio . It is used in spectral analysis and digital image processing .

Applications in economics

A relatively new field of application of the MEM is macroeconomics. The use of the MEM came into being within the framework of the economic physical trend , which, apart from the economic mainstream, applies various methods of statistical mechanics to the modeling of the economy.

Applications in ecology

In biogeography , the principle of maximum entropy is the modeling of distribution areas used. One example of this is the Maxent software .

literature

  • Edwin Thompson Jaynes: Information Theory and Statistical Mechanics . In: The Physical Review . tape 106 , no. 4 , May 15, 1957, pp. 620–630 ( bayes.wustl.edu [PDF]).
  • Nailong Wu: The Maximum Entropy Method . Springer, Berlin 1997, ISBN 978-3-540-61965-9 .

Web links

Individual evidence

  1. ^ Edwin Thompson Jaynes: Information Theory and Statistical Mechanics . In: The Physical Review . tape 106 , no. 4 , May 15, 1957, pp. 620–630 ( bayes.wustl.edu [PDF]).
  2. ↑ Persi Diaconis: A Frequentist Does This, A Bayesian That . In: SIAM News . March 13, 2004 ( full text [accessed December 28, 2007]). full text ( memento of the original from October 7, 2007 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice.  @1@ 2Template: Webachiv / IABot / www.siam.org
  3. ^ Duncan K. Foley: Statistical Equilibrium in Economics: Method, Interpretation, and an Example ( Memento of September 8, 2006 in the Internet Archive ) In: XII Workshop on "General Equilibrium: Problems, Prospects and Alternatives" 07-1999 New School University , New York.
  4. ↑ Steven J. Phillips, Miroslav Dudík, Robert E. Schapire (2006): Maximum entropy modeling of species geographic distributions . Ecological Modeling 190, 231-259. pdf