Frobenius norm

The Frobenius norm or Schur standard (named after Ferdinand Georg Frobenius or Issai Schur ) is in the mathematics one on the Euclidean norm based matrix norm . It is defined as the root of the sum of the squares of the absolute values ​​of all matrix elements. There are a number of other representations for the Frobenius norm, for example using a trace , using a scalar product , using a singular value decomposition or using a Schur decomposition . The Frobenius norm is sub-multiplicative , compatible with the Euclidean vector norm and invariant under unitary transformations , but it is not an operator norm . It is used, for example, in numerical linear algebra because of its simpler computability to estimate the spectral norm , and it is used to solve linear compensation problems by means of the Moore-Penrose inverse .

definition

The Frobenius norm of a real or complex ( m  ×  n ) matrix with out the field of real or complex numbers is defined as ${\displaystyle \|\cdot \|_{F}}$ ${\displaystyle A\in {\mathbb {K} }^{m\times n}}$ ${\displaystyle \mathbb {K} }$

${\displaystyle \|A\|_{F}:={\sqrt {\sum _{i=1}^{m}\sum _{j=1}^{n}|a_{ij}|^{2}}}}$,

thus the root of the sum of the squares of the absolute values ​​of all matrix elements . The Frobenius norm thus corresponds to the Euclidean norm of a vector of length in which all entries of the matrix are noted below one another. In the real case, the amount bars can also be omitted in the definition, but not in the complex case. ${\displaystyle a_{ij}}$ ${\displaystyle m\cdot n}$

The Frobenius norm is named after the German mathematician Ferdinand Georg Frobenius . It is also called Schurnorm after his pupil Issai Schur and is sometimes also called Hilbert-Schmidt-Norm (after David Hilbert and Erhard Schmidt ), the latter name mostly being used when investigating certain linear mappings on (possibly infinite-dimensional) Hilbert spaces , see Hilbert- Schmidt operator .

Examples

Real matrix

The Frobenius norm of the real (3 × 3) matrix

${\displaystyle A={\begin{pmatrix}1&2&1\\-1&2&-3\\0&1&-2\\\end{pmatrix}}}$

is given as

${\displaystyle \|A\|_{F}={\sqrt {\sum _{i=1}^{3}\sum _{j=1}^{3}(a_{ij})^{2}}}={\sqrt {1^{2}+2^{2}+1^{2}+({-}1)^{2}+2^{2}+({-}3)^{2}+0^{2}+1^{2}+({-}2)^{2}}}={\sqrt {25}}=5}$.

Complex matrix

The Frobenius norm of the complex (2 × 2) matrix

${\displaystyle A={\begin{pmatrix}1&i\\-2i&3-i\\\end{pmatrix}}}$

is given as

${\displaystyle \|A\|_{F}={\sqrt {\sum _{i=1}^{2}\sum _{j=1}^{2}|a_{ij}|^{2}}}={\sqrt {|1|^{2}+|i|^{2}+|{-}2i|^{2}+|3-i|^{2}}}={\sqrt {1^{2}+1^{2}+2^{2}+(3^{2}+1^{2})}}={\sqrt {16}}=4}$.

Further representations

Representation via a track

If the adjoint matrix (in the real case transposed matrix ) is , then applies to the trace (the sum of the diagonal entries) of the matrix product ${\displaystyle A^{H}\in {\mathbb {K} }^{n\times m}}$ ${\displaystyle A\in {\mathbb {K} }^{m\times n}}$ ${\displaystyle A^{H}A}$

${\displaystyle \operatorname {spur} (A^{H}A)=\sum _{i=1}^{m}\sum _{k=1}^{n}{\bar {a}}_{ik}\cdot a_{ik}=\sum _{i=1}^{m}\sum _{k=1}^{n}|a_{ik}|^{2}=\|A\|_{F}^{2}}$.

Thus the Frobenius norm has the representation

${\displaystyle \|A\|_{F}={\sqrt {\operatorname {spur} \left(A^{H}A\right)}}={\sqrt {\operatorname {spur} \left(AA^{H}\right)}}=\|A^{H}\|_{F}}$

where the middle equation follows from the fact that matrices may be interchanged cyclically under the track . The Frobenius norm is thus self adjoint .

Representation via a scalar product

On the matrix space of real or complex ( m  ×  n ) matrices defined for ${\displaystyle A,B\in \mathbb {K} ^{m\times n}}$

${\displaystyle \langle A,B\rangle =\operatorname {spur} \left(A^{H}B\right)}$

a scalar product , which is also called the Frobenius scalar product . Thus the Frobenius norm is the norm induced by the Frobenius scalar product

${\displaystyle \|A\|_{F}={\sqrt {\langle A,A\rangle }}}$.

The space of real or complex matrices is a Hilbert space with this scalar product and a Banach space with the Frobenius norm .

Representation via a singular value decomposition

If one considers a singular value decomposition of the matrix ${\displaystyle A\in {\mathbb {K} }^{m\times n}}$

${\displaystyle A=U\Sigma V^{H}}$

into a unitary matrix , a real diagonal matrix and an adjoint unitary matrix , then ${\displaystyle U\in {\mathbb {K} }^{m\times m}}$ ${\displaystyle \Sigma \in {\mathbb {R} }^{m\times n}}$ ${\displaystyle V^{H}\in {\mathbb {K} }^{n\times n}}$

${\displaystyle \operatorname {spur} \left(A^{H}A\right)=\operatorname {spur} \left(\left(V\Sigma ^{H}U^{H}\right)\left(U\Sigma V^{H}\right)\right)=\operatorname {spur} \left(V\Sigma ^{H}\Sigma V^{H}\right)=\operatorname {spur} \left(\Sigma ^{H}\Sigma \right)=\sum _{i=1}^{r}\sigma _{i}^{2}}$,

where with are the positive entries of the diagonal matrix . These entries are the singular values ​​of and equal to the square roots of the eigenvalues of . Thus the Frobenius norm has the representation ${\displaystyle \sigma _{1},\ldots ,\sigma _{r}}$ ${\displaystyle r=\operatorname {rang} (A)}$ ${\displaystyle \Sigma }$ ${\displaystyle A}$ ${\displaystyle A^{H}A}$

${\displaystyle \|A\|_{F}={\sqrt {\sigma _{1}^{2}+\ldots +\sigma _{r}^{2}}}}$,

with which it corresponds to the Euclidean norm of the vector of singular values ​​and thus the shadow-2 norm .

Representation of a Schur dismantling

If one continues to consider a Schur decomposition of a square matrix ${\displaystyle A\in {\mathbb {K} }^{n\times n}}$

${\displaystyle A=URU^{H}}$

into a unitary matrix , an upper triangular matrix and the matrix to be adjoint , then applies ${\displaystyle U\in {\mathbb {K} }^{n\times n}}$ ${\displaystyle R\in {\mathbb {K} }^{n\times n}}$ ${\displaystyle U}$ ${\displaystyle U^{H}}$

${\displaystyle \operatorname {spur} \left(A^{H}A\right)=\operatorname {spur} \left(\left(UR^{H}U^{H}\right)\left(URU^{H}\right)\right)=\operatorname {spur} \left(UR^{H}RU^{H}\right)=\operatorname {spur} \left(R^{H}R\right)=\|R\|_{F}^{2}}$.

If the matrix is ​​broken down into its main diagonal consisting of the eigenvalues of and a strictly upper triangular matrix , then applies to the Frobenius norm of ${\displaystyle R}$ ${\displaystyle \Lambda \in {\mathbb {K} }^{n\times n}}$ ${\displaystyle \lambda _{1},\ldots ,\lambda _{n}}$ ${\displaystyle A}$ ${\displaystyle N\in {\mathbb {K} }^{n\times n}}$ ${\displaystyle A}$

${\displaystyle \|A\|_{F}=\|R\|_{F}={\sqrt {\|\Lambda \|_{F}^{2}+\|N\|_{F}^{2}}}={\sqrt {(|\lambda _{1}|^{2}+\ldots +|\lambda _{n}|^{2})+\|N\|_{F}^{2}}}}$,

where the Frobenius norm is zero if and only if is a normal matrix . Is not normal, then represents a measure of the deviation from normality. ${\displaystyle N}$ ${\displaystyle A}$ ${\displaystyle A}$ ${\displaystyle \|N\|_{F}}$

properties

Standard properties

Since the sum of two matrices and the multiplication of a matrix with a scalar are defined component-wise, the norm properties of definiteness , absolute homogeneity and subadditivity follow directly from the corresponding properties of the Euclidean norm. In particular, the triangle inequality follows ${\displaystyle A,B\in {\mathbb {K} }^{m\times n}}$

${\displaystyle \|A+B\|_{F}\leq \|A\|_{F}+\|B\|_{F}}$

from the Cauchy-Schwarz inequality via

${\displaystyle \|A+B\|_{F}^{2}=\|A\|_{F}^{2}+2\operatorname {Re} \langle A,B\rangle +\|B\|_{F}^{2}\leq \|A\|_{F}^{2}+2\|A\|_{F}\|B\|_{F}+\|B\|_{F}^{2}=\left(\|A\|_{F}+\!\|B\|_{F}\right)^{2}}$,

where the above scalar product is on matrices and indicates the real part of the complex number. ${\displaystyle \langle \cdot ,\cdot \rangle }$ ${\displaystyle \operatorname {Re} }$

Sub-multiplicativity

The Frobenius norm is sub-multiplicative , that is, for matrices and holds ${\displaystyle A\in {\mathbb {K} }^{m\times n}}$ ${\displaystyle B\in {\mathbb {K} }^{n\times l}}$

${\displaystyle \|A\,B\|_{F}\leq \|A\|_{F}\,\|B\|_{F}}$,

as also with the help of the Cauchy-Schwarz inequality

${\displaystyle {\begin{aligned}\|A\,B\|_{F}^{2}&=\sum _{i=1}^{m}\sum _{k=1}^{l}\left|\sum _{j=1}^{n}a_{ij}b_{jk}\right|^{2}\!=\sum _{i=1}^{m}\sum _{k=1}^{l}\left|\langle a_{i\ast }^{H},b_{\ast k}\rangle \right|^{2}\leq \sum _{i=1}^{m}\sum _{k=1}^{l}\|a_{i\ast }^{H}\|_{2}^{2}\,\|b_{\ast k}\|_{2}^{2}\\&=\sum _{i=1}^{m}\|a_{i\ast }^{H}\|_{2}^{2}\,\sum _{k=1}^{l}\|b_{\ast k}\|_{2}^{2}=\|A\|_{F}^{2}\,\|B\|_{F}^{2}\end{aligned}}}$

can be shown. Here the -th row is from , the -th column from , the standard scalar product on vectors and the Euclidean vector norm. ${\displaystyle a_{i\ast }}$ ${\displaystyle i}$ ${\displaystyle A}$ ${\displaystyle b_{\ast k}}$ ${\displaystyle k}$ ${\displaystyle B}$ ${\displaystyle \langle \cdot ,\cdot \rangle }$ ${\displaystyle \|\cdot \|_{2}}$

Compatibility with the Euclidean norm

The Frobenius norm is compatible with the Euclidean norm , that is , the inequality applies to a matrix and a vector ${\displaystyle A\in {\mathbb {K} }^{m\times n}}$ ${\displaystyle x\in {\mathbb {K} }^{n}}$

${\displaystyle \|A\,x\|_{2}\leq \|A\|_{F}\,\|x\|_{2}}$,

which in turn comes from the Cauchy-Schwarz inequality

${\displaystyle \|A\,x\|_{2}^{2}=\sum _{i=1}^{m}\left|\sum _{j=1}^{n}a_{ij}x_{j}\right|^{2}=\sum _{i=1}^{m}\left|\langle a_{i\ast }^{H},x\rangle \right|^{2}\leq \sum _{i=1}^{m}\|a_{i\ast }^{H}\|_{2}^{2}\,\|x\|_{2}^{2}=\|A\|_{F}^{2}\,\|x\|_{2}^{2}}$

follows and which only represents the special case of submultiplicativity for . ${\displaystyle l=1}$

Unitary invariance

The Frobenius norm is invariant under unitary transformations (in the real case orthogonal transformations ), that is

${\displaystyle \|UAV\|_{F}=\|A\|_{F}}$

for all unitary matrices and . This follows directly from the track display ${\displaystyle U\in {\mathbb {K} }^{m\times m}}$ ${\displaystyle V\in {\mathbb {K} }^{n\times n}}$

${\displaystyle \|UAV\|_{F}^{2}=\operatorname {spur} \left(\left(V^{H}A^{H}U^{H}\right)\left(UAV\right)\right)=\operatorname {spur} \left(A^{H}A\right)=\|A\|_{F}^{2}}$.

Due to this invariance, the condition of a matrix with respect to the Frobenius norm does not change after a multiplication with a unitary matrix from left or right.

Cannot be represented as an operator norm

The Frobenius norm is not an operator norm and therefore not a natural matrix norm , that is, there is no vector norm , so ${\displaystyle \|\cdot \|}$

${\displaystyle \max _{x\neq 0}{\frac {\|Ax\|}{\|x\|}}=\|A\|_{F}}$

applies, as each operator norm for the identity matrix the value one must possess, but for greater results in a value less than one. Even a correspondingly scaled version of the Frobenius norm is not an operator norm, since this norm is then not sub-multiplicative, which is another property of every operator norm. ${\displaystyle I}$ ${\displaystyle \|I\|_{F}={\sqrt {\min\{m,n\}}}}$ ${\displaystyle m,n\geq 2}$

Special cases

Normal matrices

If the matrix is normal with eigenvalues , then applies ${\displaystyle A\in {\mathbb {K} }^{n\times n}}$ ${\displaystyle \lambda _{1},\ldots ,\lambda _{n}}$

${\displaystyle \|A\|_{F}={\sqrt {|\lambda _{1}|^{2}+\ldots +|\lambda _{n}|^{2}}}}$.

The Frobenius norm thus corresponds to the Euclidean norm of the vector of the eigenvalues ​​of the matrix.

Unitary matrices

If the matrix is unitary (in the real case orthogonal), then applies ${\displaystyle A\in {\mathbb {K} }^{n\times n}}$

${\displaystyle \|A\|_{F}={\sqrt {\operatorname {spur} \left(A^{H}A\right)}}={\sqrt {\operatorname {spur} \left(I\right)}}={\sqrt {n}}}$.

In this case, the Frobenius norm only depends on the size of the matrix.

Rank one matrices

Has the matrix to rank zero or one, that is with and , then applies ${\displaystyle A\in {\mathbb {K} }^{m\times n}}$ ${\displaystyle A=xy^{T}}$ ${\displaystyle x\in {\mathbb {K} }^{m}}$ ${\displaystyle y\in {\mathbb {K} }^{n}}$

${\displaystyle \|A\|_{F}=\|x\|_{2}\cdot \|y\|_{2}}$,

where again is the Euclidean vector norm. ${\displaystyle \|\cdot \|_{2}}$

Applications

Estimation of the spectral norm

The Frobenius norm is often used in numerical linear algebra to estimate the spectral norm because it is easier to calculate

${\displaystyle \|A\|_{2}\leq \|A\|_{F}\leq {\sqrt {\min\{m,n\}}}\cdot \|A\|_{2}}$.

Equality applies if and only if the rank of the matrix is ​​zero or one. These two estimates follow from the representation of the Frobenius norm via the singular value decomposition

${\displaystyle \|A\|_{2}^{2}=\sigma _{\max }^{2}\leq \sigma _{1}^{2}+\ldots +\sigma _{r}^{2}=\|A\|_{F}^{2}=\sigma _{1}^{2}+\ldots +\sigma _{r}^{2}\leq r\cdot \sigma _{\max }^{2}=r\cdot \|A\|_{2}^{2}}$,

where with are the singular values ​​of and is the maximum singular value of which just corresponds to the spectral standard. The sum of the squares of the singular values ​​is estimated by the square of the largest singular value downwards and by r times the square of the largest singular value upwards. ${\displaystyle \sigma _{1},\ldots ,\sigma _{r}}$ ${\displaystyle r\leq \min\{m,n\}}$ ${\displaystyle A}$ ${\displaystyle \sigma _{\max }}$ ${\displaystyle A}$

Linear compensation problems

If a singular or non-square matrix is, the question often arises as to its approximate inverse , i.e. a matrix , so that ${\displaystyle A}$ ${\displaystyle Z}$

${\displaystyle A\cdot Z\approx I}$.

with as the identity matrix. The Moore-Penrose inverse is an important such pseudo inverse and is defined as the matrix for which the deviation in the Frobenius norm ${\displaystyle I}$ ${\displaystyle A^{+}}$

${\displaystyle \|I-A\cdot Z\|_{F}}$

becomes minimal. She means of a singular value decomposition of the representation ${\displaystyle A}$

${\displaystyle A^{+}=V\Sigma ^{+}U^{H}}$,

where the diagonal matrix arises from the fact that the non-zero elements are inverted. Matrix equations , for example, can be set using a pseudo inverse ${\displaystyle \Sigma ^{+}}$ ${\displaystyle \Sigma }$

${\displaystyle A\cdot X=B}$

by

${\displaystyle X\approx A^{+}B}$

solve approximately, where the approximate solution via the Moore-Penrose inverse then the error

${\displaystyle \|B-A\cdot X\|_{F}}$

minimized in the Frobenius norm in terms of the least squares method .

literature

• Gene Golub , Charles van Loan: Matrix Computations . 3. Edition. Johns Hopkins University Press, 1996, ISBN 978-0-8018-5414-9 .
• Roger Horn, Charles R. Johnson: Matrix Analysis . Cambridge University Press, 1990, ISBN 978-0-521-38632-6 .
• Hans Rudolf Schwarz, Norbert Köckler: Numerical Mathematics . 8th edition. Vieweg & Teubner, 2011, ISBN 978-3-8348-1551-4 .

Web links

• Eric W. Weisstein : Frobenius Norm . In: MathWorld (English).
• Cam McLeman, Logan Hanks: Frobenius matrix norm . In: PlanetMath . (English)