Hadamard product

This article covers the Hadamard product of matrices; for the product of the same name in number theory see Riemann function # product development .

In the Hadamard product, all matrices involved have the same number of columns and rows.

The Hadamard product , Schur product , component-wise product or element-wise product is a special product of two matrices of the same size in mathematics . The resulting matrix is ​​obtained by multiplying the corresponding entries of the two output matrices . The Hadamard product is associative , with matrix addition distributive and commutative if the underlying ring is also commutative.

The Hadamard product has some interesting properties. For example, the Hadamard product of two positively semidefinite matrices is again positively semidefinite. Various parameters (such as norm , rank or spectral radius ) of the Hadamard product can also be estimated using the product of the respective parameters of the output matrices. In comparison to the more complex matrix product, the Hadamard product is less important in practice. It is named after the mathematicians Jacques Hadamard and Issai Schur .

definition

When calculating the Hadamard product, the matrix entries are multiplied with one another component by component.

If a ring is and are and two matrices are over , then the Hadamard product of and through ${\displaystyle (R,+,\cdot )}$ ${\displaystyle A=(a_{ij})\in R^{m\times n}}$ ${\displaystyle B=(b_{ij})\in R^{m\times n}}$ ${\displaystyle R}$ ${\displaystyle A}$ ${\displaystyle B}$

${\displaystyle A\circ B=(a_{ij}\cdot b_{ij})={\begin{pmatrix}a_{11}\cdot b_{11}&\cdots &a_{1n}\cdot b_{1n}\\\vdots &\ddots &\vdots \\a_{m1}\cdot b_{m1}&\cdots &a_{mn}\cdot b_{mn}\end{pmatrix}}\in R^{m\times n}}$

Are defined. The result is a matrix of the same size, with each entry being calculated by component-wise multiplication of the entries in the matrix with the entries in the matrix . The sign is occasionally used as an operator symbol for the Hadamard product . ${\displaystyle A}$ ${\displaystyle B}$ ${\displaystyle \odot }$

example

The Hadamard product of the two real (2 × 2) matrices

${\displaystyle A={\begin{pmatrix}3&2\\0&1\end{pmatrix}}}$   and   ${\displaystyle B={\begin{pmatrix}1&2\\3&1\end{pmatrix}}}$

is given by

${\displaystyle A\circ B={\begin{pmatrix}3\cdot 1&2\cdot 2\\0\cdot 3&1\cdot 1\end{pmatrix}}={\begin{pmatrix}3&4\\0&1\end{pmatrix}}}$.

properties

Laws of Calculation

The Hadamard product essentially inherits the properties of the underlying ring. It is always associative , ie for matrices applies ${\displaystyle A,B,C\in R^{m\times n}}$

${\displaystyle A\circ (B\circ C)=(A\circ B)\circ C}$,

and it is compatible with the multiplication of scalars , so ${\displaystyle a\in R}$

${\displaystyle a\,(A\circ B)=(a\,A)\circ B=A\circ (a\,B)}$.

If the underlying ring is commutative , then the Hadamard product is also commutative, that is

${\displaystyle A\circ B=B\circ A}$,

where it differs from the die product normally used . With the component-wise matrix addition, the distributive laws also apply ${\displaystyle A+B}$

${\displaystyle (A+B)\circ C=A\circ C+B\circ C}$   and   . ${\displaystyle A\circ (B+C)=A\circ B+A\circ C}$

The following also applies to the transposed matrix of a Hadamard product

${\displaystyle (A\circ B)^{T}=A^{T}\circ B^{T}}$.

The Hadamard product of two symmetric matrices is therefore symmetric again.

Algebraic structures

The number of matrices above a ring forms a ring again with the matrix addition and the Hadamard product . If a unitary ring with a one element , then the matrix ring also has a one element, the one matrix , in which all elements are the same . With the one matrix then applies to all matrices ${\displaystyle (R^{m\times n},+,\circ )}$ ${\displaystyle R}$ ${\displaystyle 1}$ ${\displaystyle J\in R^{m\times n}}$ ${\displaystyle 1}$ ${\displaystyle A\in R^{m\times n}}$

${\displaystyle A\circ J=J\circ A=A}$.

If there is a field , then a matrix is ​​called Hadamard-invertible if all entries are not equal to the zero element . The set of Hadamard invertible matrices then forms a group , with the entries being the Hadamard inverses of by ${\displaystyle R}$ ${\displaystyle A}$ ${\displaystyle A}$ ${\displaystyle (R^{m\times n},\circ )}$ ${\displaystyle {\hat {A}}=({\hat {a}}_{ij})}$ ${\displaystyle A}$

${\displaystyle {\hat {a}}_{ij}=a_{ij}^{-1}}$

given are. In the following, only matrices over the field of real or complex numbers are considered. ${\displaystyle \mathbb {K} }$

Positive definiteness

If the square matrices are positive semidefinite , then their Hadamard product is also positive semidefinite and holds for the eigenvalues ​​of ${\displaystyle A,B\in \mathbb {K} ^{n\times n}}$ ${\displaystyle A\circ B}$ ${\displaystyle A\circ B}$

${\displaystyle \left(\min _{1\leq i\leq n}a_{ii}\right)\lambda _{\min }(B)\leq \lambda {_{\min }}(A\circ B)\leq \lambda {_{\max }}(A\circ B)\leq \left(\max _{1\leq i\leq n}a_{ii}\right)\lambda {_{\max }}(B)}$.

If positive is definite and positive is semidefinite with positive main diagonal entries , then the Hadamard product is also positive definite. These statements go back to Issai Schur , who first formulated them in 1911. ${\displaystyle B}$ ${\displaystyle A}$ ${\displaystyle A\circ B}$

Estimates

Spectral standard

If the square matrix is positive definite, then the spectral norm of a Hadamard product applies ${\displaystyle A\in \mathbb {K} ^{n\times n}}$

${\displaystyle \|A\circ B\|_{2}\leq \left(\max _{1\leq i\leq n}a_{ii}\right)\|B\|_{2}}$.

If the product of two matrices then holds ${\displaystyle A=X^{H}Y}$

${\displaystyle \|A\circ B\|_{2}\leq c(X)c(Y)\|B\|_{2}}$,

where is the maximum Euclidean norm of the column vectors of . Overall, one obtains the following estimate: ${\displaystyle c(Z)}$ ${\displaystyle Z}$

${\displaystyle \|A\circ B\|_{2}\leq \|A\|_{2}\cdot \|B\|_{2}}$.

These three estimates can also be traced back to Issai Schur.

Kronecker product

The result of the Kronecker product is a large matrix, which is created by considering all possible products of entries in the two output matrices. If the matrices are , then the entries of the Hadamard product can be found exactly at the intersection of the columns with the rows of the corresponding Kronecker product. The Hadamard product is thus a sub-matrix of the Kronecker product. Therefore, the spectral standard of a Hadamard product applies ${\displaystyle A\otimes B}$ ${\displaystyle A,B\in \mathbb {K} ^{m\times n}}$ ${\displaystyle A\circ B}$ ${\displaystyle 1,n+2,2n+3,\ldots ,n^{2}}$ ${\displaystyle 1,m+2,2m+3,\ldots ,m^{2}}$

${\displaystyle \|A\circ B\|_{2}\leq \|A\otimes B\|_{2}=\|A\|_{2}\cdot \|B\|_{2}}$

and for the rank of Hadamard product

${\displaystyle \operatorname {rang} (A\circ B)\leq \operatorname {rang} (A\otimes B)=\operatorname {rang} (A)\cdot \operatorname {rang} (B)}$.

If you have two matrices and only non-negative entries, then this also applies to and . If there and are square, then the following applies to the spectral radius (the absolute value of the greatest eigenvalue ) of a Hadamard product ${\displaystyle A}$ ${\displaystyle B}$ ${\displaystyle A\circ B}$ ${\displaystyle A\otimes B}$ ${\displaystyle A}$ ${\displaystyle B}$

${\displaystyle \rho (A\circ B)\leq \rho (A\otimes B)=\rho (A)\cdot \rho (B)}$.

Induced sesquilinear form

For diagonal matrices (and only for these) the Hadamard product and the normal matrix product are the same: ${\displaystyle A,B\in \mathbb {K} ^{n\times n}}$

${\displaystyle A\circ B=A\cdot B}$.

If there are any two (column) vectors and two diagonal matrices with entries from and on the diagonal, then applies ${\displaystyle A,B\in \mathbb {K} ^{n\times n}}$ ${\displaystyle x,y\in \mathbb {K} ^{n}}$ ${\displaystyle D_{x},D_{y}\in \mathbb {K} ^{n\times n}}$ ${\displaystyle x}$ ${\displaystyle y}$

${\displaystyle x^{H}(A\circ B)y=\operatorname {spur} (D_{x}^{H}\cdot A\cdot D_{y}\cdot B^{T})}$.

Thus, the sesquilinear form produced by the Hadamard product can be written as a track . From this, for example, the sub-multiplicativity of the Frobenius norm with respect to the Hadamard product follows:

${\displaystyle \|A\circ B\|_{F}\leq \|A\|_{F}\cdot \|B\|_{F}}$.

programming

The Hadamard product is integrated into programming systems in different ways:

• In the numerical software package Matlab , the Hadamard product is represented by the symbol combination .*, while stands *for the matrix product .
• In the Fortran programming language , the Hadamard product is *implemented using the simple multiplication operator , while a separate routine matmulis available for matrix multiplication . These names were also adopted for NumPy in Python .
• In the statistical software R , the Hadamard product is *represented by, while the matrix multiplication is %*%implemented by.

literature

• Roger A. Horn: The Hadamard Product . In: Charles R. Johnson (Ed.): Matrix Theory and Applications (=  Proceedings of Symposia in Applied Mathematics . Volume 40 ). American Mathematical Society, Providence RI 1990, ISBN 0-8218-0154-6 , pp. 87-170 .
• Roger A. Horn, Charles R. Johnson: Topics in Matrix Analysis. 1st paperback edition. Cambridge University Press, Cambridge et al. 1994, ISBN 0-521-46713-6 , pp. 298-381.

Individual evidence

1. ↑ ^{a b c} Horn: The Hadamard Product. In: Johnson (Ed.): Matrix Theory and Applications. 1990, pp. 87-170, here p. 88.
2. ^ Christian Voigt, Jürgen Adamy: Collection of formulas for the matrix calculation . Oldenbourg, Munich et al. 2007, ISBN 978-3-486-58350-2 , pp. 13 .
3. ↑ ^{a b} Horn: The Hadamard Product. In: Johnson (Ed.): Matrix Theory and Applications. 1990, pp. 87-170, here p. 95.
4. ↑ ^{a b c} Horn: The Hadamard Product. In: Johnson (Ed.): Matrix Theory and Applications. 1990, pp. 87-170, here pp. 96-100.
5. ↑ ^{a b} Horn: The Hadamard Product. In: Johnson (Ed.): Matrix Theory and Applications. 1990, pp. 87-170, here pp. 100-104.
6. ↑ Christoph Überhuber , Stefan Katzenbeisser, Dirk Praetorius: MATLAB 7. An introduction . Springer, Vienna et al. 2005, ISBN 3-211-21137-3 , p. 81 .