Whitening (statistics)

The whitening transformation describes a linear transformation in which a vector of random variables with a known covariance matrix is ​​converted into a series of new variables whose covariance matrix is similar to the identity matrix . The aim is to achieve a state in which the individual variables are uncorrelated and each have a variance of 1. The transformation is called “whitening”, since after the conversion the distribution properties of the input vector correspond to those of the white noise . The aim is constant equal distribution .

Some other transformations are closely related to whitening:

1. the decorrelation transformation only removes correlations but leaves the variances intact,
2. the standardization transformation sets the variances 1 but leaves the correlations intact,
3. the coloring transformation translates a vector with "white" random variables into a random vector with a specific covariance matrix.

definition

Assuming a random column vector with non-singular covariance matrix and mean . Then the transformation with a whitening matrix that fulfills the condition leads to a “whitened” random vector with a uniform diagonal covariance. ${\displaystyle X}$ ${\displaystyle M}$ ${\displaystyle 0}$ ${\displaystyle Y=WX}$ ${\displaystyle W}$ ${\displaystyle W^{\mathrm {T} }W=M^{-1}}$ ${\displaystyle Y}$

There are an infinite number of possible whitening matrices . Common choices are ( Mahalanobis or ZCA whitening), the Cholesky decomposition of (Cholesky whitening), or the eigenvectors of (PCA whitening). ${\displaystyle W}$ ${\displaystyle W=M^{-1/2}}$ ${\displaystyle M^{-1}}$ ${\displaystyle M}$

Kessy et al. (2018) demonstrate that optimal whitening transformations can be identified by examining the cross-variances and cross-correlations of  and . For example, the singular optimal whitening transformation for achieving the maximum component-wise correlation between the original and the whitened is generated by the whitening matrix . Here is the correlation matrix and the variance matrix. ${\displaystyle X}$ ${\displaystyle Y}$ ${\displaystyle X}$ ${\displaystyle Y}$ ${\displaystyle W=P^{-1/2}V^{-1/2}}$ ${\displaystyle P}$ ${\displaystyle V}$

A data matrix "white"

The whitening of a data matrix follows the same transformations as random variables. An empirical whitening transformation is carried out by estimating the covariance (e.g. using the maximum likelihood method ) and then constructing a correspondingly estimated whitening matrix (e.g. using the Cholesky decomposition ).

See also

• Principal component analysis
• Canonical correlation

Web links

• courses.media.mit.edu (PDF)
• The ZCA whitening transformation. (PDF) Appendix A by A. Krizhevsky: Learning Multiple Layers of Features from Tiny Images.

Individual evidence

1. ↑ Agnan Kessy, Alex Lewin, Korbinian Strimmer: Optimal Whitening and Decorrelation . In: The American Statistician . 2017, ISSN  0003-1305 , p. 1–6 , doi : 10.1080 / 00031305.2016.1277159 , arxiv : 1512.00809 (as of 2018).
2. ↑ Miliha Hossain: Whitening and Coloring Transforms for Multivariate Gaussian Random Variables . Project Rhea. Retrieved March 21, 2016.
3. ^ Jerome H. Friedman: Exploratory Projection Pursuit . In: Journal of the American Statistical Association . tape 82 , no. 397 , March 1987, p. 249 , doi : 10.2307 / 2289161 , JSTOR : 2289161 .