Hessian matrix

The Hesse matrix named after Otto Hesse is a matrix that is an analogue of the second derivative of a function in multi-dimensional real analysis .

The Hesse matrix appears in the approximation of a multidimensional function in the Taylor expansion . Among other things, it is important in connection with the optimization of systems that are described by several parameters, as they often occur in economics, physics, theoretical chemistry or engineering.

definition

Let be a twice continuously differentiable function . Then the Hessian matrix of at point is defined by ${\displaystyle f\colon D\subset \mathbb {R} ^{n}\to \mathbb {R} }$ ${\displaystyle f}$ ${\displaystyle x=(x_{1},\ldots ,x_{n})\in D}$

${\displaystyle \operatorname {H} _{f}(x):=\left({\frac {\partial ^{2}f}{\partial x_{i}\partial x_{j}}}(x)\right)_{i,j=1,\dots ,n}={\begin{pmatrix}{\frac {\partial ^{2}f}{\partial x_{1}\partial x_{1}}}(x)&{\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}}(x)&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\partial x_{n}}}(x)\\[0.5em]{\frac {\partial ^{2}f}{\partial x_{2}\partial x_{1}}}(x)&{\frac {\partial ^{2}f}{\partial x_{2}\partial x_{2}}}(x)&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\partial x_{n}}}(x)\\\vdots &\vdots &\ddots &\vdots \\{\frac {\partial ^{2}f}{\partial x_{n}\partial x_{1}}}(x)&{\frac {\partial ^{2}f}{\partial x_{n}\partial x_{2}}}(x)&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}\partial x_{n}}}(x)\end{pmatrix}}.}$

With that are second partial derivatives designated. The Hessian matrix corresponds to the transpose of the Jacobian matrix of the gradient , but is symmetrical in the case of continuous second derivatives because of the interchangeability of the differentiation order , so that transposing the matrix does not cause any change. ${\displaystyle {\tfrac {\partial ^{2}f}{\partial x_{i}\partial x_{j}}}}$

Examples

• For , applies and , thus ${\displaystyle f\colon \mathbb {R} ^{2}\to \mathbb {R} }$ ${\displaystyle f(x,y)=x^{3}+y^{3}-3xy}$ ${\displaystyle {\frac {\partial f}{\partial x}}(x,y)=3x^{2}-3y}$ ${\displaystyle {\frac {\partial f}{\partial y}}(x,y)=3y^{2}-3x}$

${\displaystyle \operatorname {H} _{f}(x,y)={\begin{pmatrix}6x&-3\\-3&6y\end{pmatrix}}}$.

• The function , which each vector in its Euclidean norm assigns, for all continuously differentiable twice, and it is according to the chain rule ${\displaystyle r\colon \mathbb {R} ^{n}\to \mathbb {R} }$ ${\displaystyle r(x)=\|x\|={\sqrt {\sum _{j=1}^{n}x_{j}^{2}}}}$ ${\displaystyle \mathbb {R} ^{n}}$ ${\displaystyle x\neq 0}$

${\displaystyle {\frac {\partial r}{\partial x_{j}}}(x)={\frac {x_{j}}{\|x\|}}}$

as well as according to the quotient rule

${\displaystyle {\frac {\partial ^{2}r}{\partial x_{i}\partial x_{j}}}(x)={\frac {\delta _{ij}\|x\|-x_{j}{\frac {x_{i}}{\|x\|}}}{\|x\|^{2}}}={\frac {1}{\|x\|}}\delta _{ij}-{\frac {x_{i}x_{j}}{\|x\|^{3}}}}$,

where the Kronecker delta denotes. In matrix notation it follows ${\displaystyle \delta _{ij}={\frac {\partial x_{j}}{\partial x_{i}}}}$

${\displaystyle \operatorname {H} _{r}(x)={\frac {1}{\|x\|}}E_{n}-{\frac {1}{\|x\|^{3}}}xx^{T}}$

with the - identity matrix . ${\displaystyle n\times n}$ ${\displaystyle E_{n}}$

Applications

Taylor evolution

The Taylor expansion of a twice continuously differentiable function with around a development point begins with ${\displaystyle f\colon D\to \mathbb {R} }$ ${\displaystyle D\subseteq \mathbb {R} ^{n}}$ ${\displaystyle a\in D}$

${\displaystyle T(x)=f(a)+(x-a)^{T}\operatorname {grad} f(a)+{\frac {1}{2}}(x-a)^{T}\operatorname {H} _{f}(a)(x-a)+\ldots }$

The second order terms of this expansion are given by the square form , the matrix of which is the Hessian matrix evaluated at the expansion point.

Extreme values

With the help of the Hesse matrix, the character of the critical points of an image can be determined. For this purpose, the definiteness of the Hessian matrix is determined for the previously determined critical points . ${\displaystyle \mathbb {R} ^{n}}$

• If the matrix is ​​positive definite at one point, there is a local minimum of the function at this point .
• If the Hessian matrix is ​​negative definite there, it is a local maximum .
• If it is indefinite, then it is a saddle point of the function.

If the Hessian matrix is ​​only semidefinite at the examined point, this criterion fails and the character of the critical point must be determined in another way. Which of these cases is present can be decided - as described under definiteness - for example with the help of the signs of the eigenvalues ​​of the matrix or its main minors .

Example: The function has a critical point, but is neither positive nor negative definite and also not semidefinite, but indefinite. The function has no extremum at this point, but a saddle point where two contour lines intersect. ${\displaystyle f(x,y)=x^{2}-y^{2}}$ ${\displaystyle (0,0)}$ ${\displaystyle H(f)(0,0)={\begin{pmatrix}2&0\\0&-2\end{pmatrix}}}$

convexity

There is also a connection between the positive definiteness of the Hessian matrix and the convexity of a twice continuously differentiable function that is defined on an open, convex set : Such a function is convex if and only if its Hessian matrix is everywhere in positive semidefinite . If the Hessian matrix is ​​even positive definite , then the function is strictly convex. Correspondingly, the following applies: A twice continuously differentiable function is concave on its convex definition set if and only if its Hessian matrix is ​​negative semidefinite. Is the Hessian matrix even negative definite on , it is on strictly concave. ${\displaystyle f}$ ${\displaystyle D}$ ${\displaystyle D}$ ${\displaystyle D}$ ${\displaystyle D}$ ${\displaystyle f}$ ${\displaystyle D}$ ${\displaystyle D}$ ${\displaystyle f}$ ${\displaystyle D}$

If its definition set is strictly convex, then it has at most one global minimum . Every local minimum is at the same time the (only) global minimum. If it is strictly concave, it has at most one global maximum. Each local maximum is also your (only) global maximum. ${\displaystyle f}$ ${\displaystyle D}$ ${\displaystyle f}$ ${\displaystyle D}$ ${\displaystyle f}$ ${\displaystyle f}$

Laplace operator

The Laplace operator of a twice continuously differentiable function with is equal to the trace of its Hessian matrix and therefore independent of the choice of coordinates: ${\displaystyle f\colon D\to \mathbb {R} }$ ${\displaystyle D\subseteq \mathbb {R} ^{n}}$

${\displaystyle \Delta f=\mathrm {Spur} ({H}_{f})}$

See also

• Modified Hessian matrix

Individual evidence

1. ^ Otto Forster : Analysis 2 . Differential calculus in R ⁿ , ordinary differential equations. 8th edition. Vieweg + Teubner Verlag, Wiesbaden 2008, ISBN 978-3-8348-0575-1 , p. 78 .
2. ↑ Convex Functions. (No longer available online.) P. 16 , archived from the original on November 2, 2013 ; Retrieved September 16, 2012 . Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice.  

Web links

• More on the connection between convexity and the Hessian matrix

Literature and individual references

• Konrad Königsberger : Analysis. Volume 2. 3rd revised edition. Springer-Verlag, Berlin et al. 2000, ISBN 3-540-66902-7 .