Spectral standard

Illustration of the spectral norm

The spectral norm is in the mathematics of the Euclidean norm derived natural matrix norm . The spectral norm of a matrix corresponds to its maximum singular value , i.e. the root of the greatest eigenvalue of the product of the adjoint ( transposed ) matrix with this matrix. It is sub-multiplicative , compatible with the Euclidean vector norm and invariant under unitary ( orthogonal ) transformations. The spectral norm of the inverse of a regular matrix is the reciprocal of the smallest singular value of the output matrix . If a matrix is Hermitian ( symmetrical ), then its spectral norm is equal to its spectral radius . If a matrix is unitary ( orthogonal ), then its spectral norm is equal to one .

Due to its complex calculability, the spectral norm is often estimated in practice using matrix norms that are easier to calculate . It is used in particular in linear algebra and numerical mathematics .

definition

The spectral norm of a matrix with as the field of real or complex numbers is the natural matrix norm derived from the Euclidean vector norm ${\displaystyle \|\cdot \|_{2}}$ ${\displaystyle A\in {\mathbb {K} }^{m\times n}}$ ${\displaystyle \mathbb {K} }$

${\displaystyle \|A\|_{2}:=\max _{x\neq 0}{\frac {\|Ax\|_{2}}{\|x\|_{2}}}=\max _{\|x\|_{2}=1}\|Ax\|_{2}}$.

The spectral standard thus clearly corresponds to the largest possible stretching factor that results from the application of the matrix to a vector of length one. An equivalent definition of the spectral norm is the radius of the smallest sphere that includes the unit circle after transformation through the matrix.

Representation as a maximum singular value

According to the definition, the Euclidean norm and the standard scalar product on vectors apply to the spectral norm ${\displaystyle \langle \cdot ,\cdot \rangle }$

${\displaystyle \|A\|_{2}^{2}=\max _{\|x\|_{2}=1}\|Ax\|_{2}^{2}=\max _{\|x\|_{2}=1}\langle Ax,Ax\rangle =\max _{\|x\|_{2}=1}\langle A^{H}A\,x,x\rangle }$,

where the adjoint (in the real case transposed ) matrix is to. The matrix is a positive semidefinite Hermitian (in the real case symmetrical ) matrix. Therefore, according to the spectral theorem, there is a unitary (in the real case orthogonal ) matrix , consisting of the eigenvectors of the matrix, so that where is a diagonal matrix with the always real and nonnegative eigenvalues of . With the substitution and the unitary invariance of the Euclidean vector norm then applies ${\displaystyle A^{H}}$ ${\displaystyle A}$ ${\displaystyle A^{H}A}$ ${\displaystyle U}$ ${\displaystyle U^{H}A^{H}A\,U=D}$ ${\displaystyle D}$ ${\displaystyle \mu _{1},\ldots ,\mu _{n}}$ ${\displaystyle A^{H}A}$ ${\displaystyle y=U^{H}x}$

${\displaystyle \|A\|_{2}^{2}=\max _{\|Uy\|_{2}=1}\langle A^{H}A\,Uy,Uy\rangle =\max _{\|y\|_{2}=1}\langle D\,y,y\rangle =\max _{\|y\|_{2}=1}(\mu _{1}\,|y_{1}|^{2}+\ldots +\mu _{n}\,|y_{n}|^{2})=\mu _{\max }}$,

where is the largest of these eigenvalues, since the maximum is assumed precisely when is equal to the unit vector for the maximum eigenvalue. The spectral norm of a matrix is thus ${\displaystyle \mu _{\max }}$ ${\displaystyle y}$ ${\displaystyle A}$

${\displaystyle \|A\|_{2}={\sqrt {\mu _{\max }}}}$,

thus the root of the greatest eigenvalue of . The largest eigenvalue of a matrix is ​​also called the spectral radius , and the roots of the eigenvalues ​​of are also referred to as singular values of . The spectral norm of a matrix therefore corresponds to its maximum singular value. ${\displaystyle A^{H}A}$ ${\displaystyle A^{H}A}$ ${\displaystyle A}$

Examples

Real matrix

The spectral norm of the real (2 × 2) matrix

${\displaystyle A=\left({\begin{matrix}3&2\\-2&0\end{matrix}}\right)}$

is determined by first calculating the matrix product : ${\displaystyle A^{T}A}$

${\displaystyle A^{T}A={\begin{pmatrix}3&-2\\2&0\end{pmatrix}}\cdot {\begin{pmatrix}3&2\\-2&0\end{pmatrix}}={\begin{pmatrix}13&6\\6&4\end{pmatrix}}}$.

The eigenvalues ​​of then result as zeros of the characteristic polynomial ${\displaystyle A^{T}A}$

${\displaystyle \operatorname {det} (\mu I-A^{T}A)=(\mu -13)(\mu -4)-6^{2}=\mu ^{2}-17\mu +16}$

as

${\displaystyle \mu _{1,2}={\frac {17\pm {\sqrt {289-64}}}{2}}={\frac {17\pm 15}{2}}=\{16,1\}}$.

The spectral norm of is thus the root of the larger of these eigenvalues, i.e. ${\displaystyle A}$

${\displaystyle \|A\|_{2}={\sqrt {\mu _{1}}}=4}$.

Complex matrix

To the spectral norm of the complex (2 × 2) matrix

${\displaystyle A={\begin{pmatrix}1-i&-3i\\2i&0\end{pmatrix}}}$

to calculate, the procedure is as in the real case. The matrix is determined ${\displaystyle A^{H}A}$

${\displaystyle A^{H}A={\begin{pmatrix}1+i&-2i\\3i&0\end{pmatrix}}\cdot {\begin{pmatrix}1-i&-3i\\2i&0\end{pmatrix}}={\begin{pmatrix}6&3-3i\\3+3i&9\end{pmatrix}}}$,

whose eigenvalues ​​then extend over the zeros of

${\displaystyle \operatorname {det} (\mu I-A^{H}A)=(\mu -6)(\mu -9)-(3-3i)(3+3i)=\mu ^{2}-15\mu +36}$

as

${\displaystyle \mu _{1,2}={\frac {15\pm {\sqrt {225-144}}}{2}}={\frac {15\pm 9}{2}}=\{12,3\}}$.

surrender. The spectral norm of is thus ${\displaystyle A}$

${\displaystyle \|A\|_{2}={\sqrt {\mu _{1}}}={\sqrt {12}}\approx 3{,}46}$.

properties

Standard properties

The norm axioms definiteness , absolute homogeneity and subadditivity follow for the spectral norm directly from the corresponding properties of natural matrix norms. In particular, the spectral norm is thus also sub-multiplicative and compatible with the Euclidean norm , that is, it applies

${\displaystyle \|A\cdot x\|_{2}\leq \|A\|_{2}\cdot \|x\|_{2}}$

for all matrices and all vectors , and the spectral norm is the smallest norm with this property. ${\displaystyle A\in {\mathbb {K} }^{m\times n}}$ ${\displaystyle x\in {\mathbb {K} }^{n}}$

Self adjointness

The spectral norm is self-adjoint , that is for the adjoint matrix of a square matrix applies ${\displaystyle A^{H}}$ ${\displaystyle A\in {\mathbb {K} }^{n\times n}}$

${\displaystyle \|A^{H}\|_{2}^{2}=\max _{\|x\|_{2}=1}\langle AA^{H}\,x,x\rangle =\max _{\|x\|_{2}=1}\langle A^{H}A\,x,x\rangle =\|A\|_{2}^{2}}$,

since the matrix and the matrix have the same eigenvalues. A transposed matrix also fulfills the same identity regardless of whether the matrix is ​​real or complex. The spectral norm is therefore invariant with adjoint or transposition of the matrix. ${\displaystyle AA^{H}}$ ${\displaystyle A^{H}A}$ ${\displaystyle A^{T}}$

Unitary invariance

The spectral norm is invariant under unitary transformations (in the real case orthogonal transformations ), that is

${\displaystyle \|UAV\|_{2}=\|A\|_{2}}$

for all unitary (or orthogonal ) matrices and , because it applies with the unitary invariance of the Euclidean norm ${\displaystyle U\in {\mathbb {K} }^{m\times m}}$ ${\displaystyle V\in {\mathbb {K} }^{n\times n}}$

${\displaystyle \|UAV\|_{2}^{2}=\max _{\|x\|_{2}=1}\langle UAVx,UAVx\rangle =\max _{\|Vx\|_{2}=1}\langle AVx,AVx\rangle =\max _{\|y\|_{2}=1}\langle Ay,Ay\rangle =\|A\|_{2}^{2}}$.

Due to the unitary invariance, the condition of a matrix with respect to the spectral norm does not change after a multiplication with a unitary matrix from left or right.

Special cases

Inverse of a regular matrix

If the matrix is regular , then the spectral norm of its inverse matrix is given as, due to the symmetry ${\displaystyle A\in {\mathbb {K} }^{n\times n}}$

${\displaystyle \|A^{-1}\|_{2}^{2}=\max _{\|x\|_{2}=1}\langle (AA^{H})^{-1}x,x\rangle =\max _{\|x\|_{2}=1}\langle (A^{H}A)^{-1}x,x\rangle =\mu _{\min }^{-1}}$,

since the inverse of a matrix just has its reciprocal eigenvalues. The spectral norm of the inverse of a matrix is ​​therefore the reciprocal of the smallest singular value of the output matrix. The following applies to the spectral condition of a regular matrix

${\displaystyle \kappa _{2}(A)=\|A\|_{2}\cdot \|A^{-1}\|_{2}={\sqrt {\mu _{\max }/\mu _{\min }}}}$,

it is therefore the ratio of the largest and smallest singular value.

Hermitian matrices

If the matrix itself is Hermitian (or symmetric), then is , and there is a unitary matrix consisting of the eigenvectors of such that ${\displaystyle A\in {\mathbb {K} }^{n\times n}}$ ${\displaystyle A^{H}A=A^{2}}$ ${\displaystyle U}$ ${\displaystyle A}$

${\displaystyle \|A\|_{2}^{2}=\max _{\|x\|_{2}=1}\langle U^{H}\!A^{2}\,Ux,x\rangle =\max _{\|x\|_{2}=1}(\lambda _{1}^{2}\,|x_{1}|^{2}+\ldots +\lambda _{n}^{2}\,|x_{n}|^{2})=\lambda _{\max }^{2}}$

holds, where the eigenvalues ​​of are always real and the magnitude of these eigenvalues ​​is the largest. So the spectral norm of a Hermitian or symmetric matrix is ${\displaystyle \lambda _{1},\ldots ,\lambda _{n}}$ ${\displaystyle A}$ ${\displaystyle \lambda _{\max }}$

${\displaystyle \|A\|_{2}=|\lambda _{\max }|}$

and thus corresponds to the spectral radius of the matrix. If the matrix is ​​still positive semi-definite, then the absolute bars can be omitted, and its spectral norm is equal to its greatest eigenvalue.

Unitary matrices

If the matrix is unitary, then we have ${\displaystyle A\in {\mathbb {K} }^{n\times n}}$

${\displaystyle \|A\|_{2}^{2}=\max _{\|x\|_{2}=1}\langle A^{H}\!A\,x,x\rangle =\max _{\|x\|_{2}=1}\langle x,x\rangle =\max _{\|x\|_{2}=1}\|x\|_{2}^{2}=1}$.

The spectral norm of a unitary or orthogonal matrix is ​​therefore equal to one .

Rank one matrices

Has the matrix to rank zero or one, that is with and , then applies ${\displaystyle A\in {\mathbb {K} }^{m\times n}}$ ${\displaystyle A=xy^{T}}$ ${\displaystyle x\in {\mathbb {K} }^{m}}$ ${\displaystyle y\in {\mathbb {K} }^{n}}$

${\displaystyle \|A\|_{2}=\|x\|_{2}\cdot \|y\|_{2}}$,

there the matrix

${\displaystyle A^{H}A=(xy^{T})^{H}(xy^{T})={\bar {y}}x^{H}xy^{T}=(x^{H}x)(yy^{H})}$

also has the rank zero or one, in the latter case the only eigenvalue is not equal to zero. ${\displaystyle (x^{H}x)(y^{H}y)}$

Estimates

Since the spectral norm is difficult to calculate, especially for large matrices, it is often estimated in practice using other, easier-to-calculate matrix norms. The most important of these estimates are

${\displaystyle \|A\|_{2}\leq {\sqrt {\|A\|_{1}\cdot \|A\|_{\infty }}}}$

as the geometric mean of the column sum norm and the row sum norm and ${\displaystyle \|\cdot \|_{1}}$ ${\displaystyle \|\cdot \|_{\infty }}$

${\displaystyle \|A\|_{2}\leq \|A\|_{F}}$,

where is the Frobenius norm . ${\displaystyle \|\cdot \|_{F}}$

Remarks

1. ↑ positive semidefinite there and hermitian there ${\displaystyle x^{H}A^{H}Ax=(Ax)^{H}(Ax)\geq 0}$ ${\displaystyle (A^{H}A)^{H}=A^{H}A}$
2. ↑ The matrix can be inverted and it applies . Thus, and are similar , which is why in particular and have the same characteristic polynomial and therefore have the same eigenvalues. ${\displaystyle X:={\begin{bmatrix}I_{n}&A\\0&I_{n}\\\end{bmatrix}}}$ ${\displaystyle X^{-1}{\begin{bmatrix}AA^{H}&0\\A^{H}&0\\\end{bmatrix}}X={\begin{bmatrix}0&0\\A^{H}&A^{H}A\\\end{bmatrix}}}$ ${\displaystyle {\begin{bmatrix}AA^{H}&0\\A^{H}&0\\\end{bmatrix}}}$ ${\displaystyle {\begin{bmatrix}0&0\\A^{H}&A^{H}A\\\end{bmatrix}}}$ ${\displaystyle AA^{H}}$ ${\displaystyle A^{H}A}$

literature

• Gene Golub , Charles van Loan: Matrix Computations . 3. Edition. Johns Hopkins University Press, 1996, ISBN 978-0-8018-5414-9 .
• Roger Horn, Charles R. Johnson: Matrix Analysis . Cambridge University Press, 1990, ISBN 978-0-521-38632-6 .
• Hans Rudolf Schwarz, Norbert Köckler: Numerical Mathematics . 8th edition. Vieweg & Teubner, 2011, ISBN 978-3-8348-1551-4 .

Web links

• Eric W. Weisstein : Spectral Norm . In: MathWorld (English).