James Stein Estimator

James Stein estimators are estimators of the expected value vector of a multidimensional normal distribution . If this normal distribution is at least three-dimensional, James Stein estimates with regard to the mean square error are equally better than the arithmetic mean usually used as an estimate . In the sense of decision theory, the arithmetic mean is therefore not a permissible decision function for dimensions greater than two for the expected value vector of the normal distribution. This fact was discovered by Charles Stein in 1956 . The first James Stein estimator goes back to a work by W. James and C. Stein from 1961.

Structure of a James Stein estimator

Let be a -dimensional normally distributed vector with stochastically independent components that have the variance one. The expected value vector is to be estimated. Obviously, the arithmetic mean is used for this because it is consistently the best unbiased estimator for . A specific James Stein estimate is the following: ${\displaystyle X=(X_{1},\dots ,X_{m})^{T}}$ ${\displaystyle m}$ ${\displaystyle \operatorname {E} X=\mu }$ ${\displaystyle {\overline {X}}=({\overline {X}}_{1},\dots ,{\overline {X}}_{m})^{T}}$ ${\displaystyle \mu }$

${\displaystyle {\overline {X}}^{*}={\Bigl (}1-{\frac {m-2}{{\overline {X}}^{T}{\overline {X}}}}{\Bigr )}{\overline {X}}={\Bigl (}1-{\frac {m-2}{\sum _{j=1}^{m}{\overline {X}}_{j}^{2}}}{\Bigr )}{\overline {X}}\quad ;\quad m\geq 3}$.

${\displaystyle {\overline {X}}^{*}}$is not true to expectations . Since component-wise magnitude smaller than is, is underestimated, ie a so-called is shrinkage estimators (engl. Shrinkage ). The shrinkage factor is chosen so that the mean square error of the estimator is smaller, despite the bias , than in the case of true to expectation . The estimator is better than , but it is also not a valid estimator itself. ${\displaystyle \operatorname {E} {\overline {X}}^{*}}$ ${\displaystyle \mu }$ ${\displaystyle \mu }$ ${\displaystyle {\overline {X}}^{*}}$ ${\displaystyle {\Bigl (}1-{\frac {m-2}{{\overline {X}}^{T}{\overline {X}}}}{\Bigr )}}$ ${\displaystyle {\overline {X}}}$ ${\displaystyle {\overline {X}}^{*}}$ ${\displaystyle {\overline {X}}}$

Generalizations

The assumption "variance equals one" has been made above for the sake of simplicity. James and Stein already indicated alternatives to and extended the investigations to linear regression models with at least three regression parameters. It is difficult to find explicitly admissible estimators for , but see. ${\displaystyle {\overline {X}}^{*}}$ ${\displaystyle \mu }$

Stone paradox

The James Stein Estimator initially led to controversial discussions, and there was even talk of the Stein Paradox . This can be seen in the following example. Be three dimensional with the components ${\displaystyle X}$

${\displaystyle X_{1}=}$ Weight of a kiwi fruit from New Zealand

${\displaystyle X_{2}=}$ Weight of a banana from Brazil

${\displaystyle X_{3}=}$ Weight of an apple from Germany

We assume that it is (at least approximately) normally distributed and that (unsurprisingly) the three components are stochastically independent of one another. Then it is quite surprising that the estimate can be improved by, for example, using the independent banana and apple data to estimate the expected value of the kiwi weight in the shrinkage factor. The surprise is put into perspective, however, if one emphasizes that the "stone effect" only occurs if one absolutely (for whatever reason) wants to evaluate the estimation of the vector with a common criterion for all three components. Estimating each component individually, of course, leads to the one-dimensional case and to the fact that it is permissible, i.e. cannot be replaced by a better estimate. A good interpretation can also be achieved through empirical Bayesian arguments. ${\displaystyle X}$ ${\displaystyle {\overline {X}}}$ ${\displaystyle \mu }$ ${\displaystyle \mu _{i};\ i=1,\dots ,m}$ ${\displaystyle {\overline {X}}_{i}}$

Individual evidence

1. ↑ Stein, C. (1956), Inadmissibility of the usual estimator for the mean of a multivariate normal distribution , Proc. Third Berkeley Symp. Math. Statist. Prob. 1, 197-206,
2. ↑ ^{a b} James, W. and Stein, C. (1961), Estimation with quadratic loss , Proc. Fourth Berkeley Symp. Math. Statist. Prob. 1, 361-379
3. ↑ Anderson, TW (1984), An Introduction to Multivariate Statistical Analysis (2nd ed.), New York: John Wiley & Sons
4. ^ Brown, LD (1966), On the admissibility of invariant estimators of one or more location parameters , Annals of Mathematical Statistics, 37 (5), 1087-1136
5. ↑ Alam, K. (1973), A family of admissible minimax estimators of the mean of a multivariate normal distribution , Annals of Statistics 1, 517-525
6. ↑ Efron, B. and Morris, C. (1977), Stein's paradox in statistics , Scientific American, 236 (5), 119-127
7. ↑ Samworth, RJ (2012), Stein's Paradox , Eureka, 62: 38-41
8. ↑ Efron, B. and Morris, C. (1973). Stein's Estimation Rule and Its Competitors — An Empirical Bayes Approach . Journal of the American Statistical Association. 68 (341), 117-130