BFGS procedure
The Broyden-Fletcher-Goldfarb-Shanno (BFGS) method is a numerical method for solving non-linear optimization problems . The method was developed independently by mathematicians Broyden, Fletcher, Goldfarb and Shanno in 1970 and published in four scientific articles.
It belongs to the group of quasi-Newton methods . As such, it avoids the direct calculation of the Hessian matrix by iteratively approximating the Hessian matrix. With quadratic functions, both the Newton method and the quasi-Newton method require approx. N² function calls (if the derivatives are approximated using difference quotients); this also applies to the conjugate gradient method . However, the BFGS method has particularly proven itself in practice (e.g. it is relatively tolerant of errors in step size control).
literature
- Charles G. Broyden : The convergence of a class of double-rank minimization algorithms . In: Journal of the Institute of Mathematics and Its Applications . tape 6 , 1970, pp. 76-90 , doi : 10.1093 / imamat / 6.1.76 .
- Roger Fletcher : A New Approach to Variable Metric Algorithms . In: Computer Journal . tape 13 , no. 3 , 1970, p. 317-322 , doi : 10.1093 / comjnl / 13.3.317 .
- Donald Goldfarb : A Family of Variable Metric Updates Derived by Variational Means . In: Mathematics of Computation . tape 24 , no. 109 , 1970, pp. 23-26 , doi : 10.1090 / S0025-5718-1970-0258249-6 .
- David F. Shanno : Conditioning of quasi-Newton methods for function minimization . In: Mathematics of Computation . tape 24 , no. 111 , July 1970, p. 647-656 , doi : 10.1090 / S0025-5718-1970-0274029-X .