Baum-Welch algorithm
In computer science and in statistical calculation models, the Baum-Welch algorithm is used to find the unknown parameters of a Hidden Markov Model (HMM). It uses the forward-backward algorithm to calculate intermediate results, but is not identical to it.
The Baum-Welch algorithm is an expectation-maximizing algorithm . It calculates the maximum likelihood estimates ( maximum likelihood estimates ) and the so-called posterior mode estimates for the given parameters (transition and emission probability) of an HMM if only the emissions are given as training data.
The algorithm works in two steps: First, it calculates the forward probability and the backward probability for each state of the HMM. Second, on the basis of this first step, it calculates the frequency of the transition-emission pair values and divides this by the probability of the entire string (so-called posterior decoding ). This leads to the calculation of the expected count of the single transition emission pair. Every time a single transition is found, the value of the quotient of the transition and the probability of the entire string increases. This value can then be made the new value of the transition.
The Baum-Welch algorithm is an instance of the EM algorithm and was named after Leonard E. Baum and Lloyd R. Welch .
literature
- Lawrence R. Rabiner, Biing-Hwang Juang: An Introduction to Hidden Markov Models. In: IEEE. ASSP Magazine. Vol. 3, No. 1, January 1986, ISSN 0740-7467 , pp. 4-16, doi : 10.1109 / MASSP.1986.1165342 .
- Christopher D. Manning, Hinrich Schütze: Foundations of Statistical Natural Language Processing. MIT Press, Cambridge MA et al. 1999, ISBN 0-262-13360-1 .
- Jeff A. Bilmes: A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models . International Computer Science Institute , Berkeley CA 1998, online (PDF; 189 kB) .