Lemma of Stein

In test theory , a branch of mathematical statistics , Stein's lemma is a statement about how the probability of a type 2 error changes in Neyman-Pearson tests for larger samples.

The statement is named after Charles Stein , who proved it in 1952.

Framework

Let a statistical model be given with a simple null hypothesis and a simple alternative , for which both the probability density functions and exist and are genuinely positive. Furthermore, let the corresponding infinite product model and the projection onto the components of the product model. ${\ displaystyle (X, {\ mathcal {A}}, \ {P_ {0}, P_ {1} \})}$ ${\ displaystyle P_ {0}}$ ${\ displaystyle P_ {1}}$ ${\ displaystyle f_ {0}}$ ${\ displaystyle f_ {1}}$ ${\ displaystyle (X ^ {\ mathbb {N}}, {\ mathcal {A}} ^ {\ mathbb {N}}, \ {P_ {0} ^ {\ mathbb {N}}, P_ {1} ^ {\ mathbb {N}} \})}$ ${\ displaystyle X_ {i}}$

Be a Neyman-Pearson test at the level that only depends on. Furthermore denote ${\ displaystyle \ Phi _ {n}}$ ${\ displaystyle \ alpha}$ ${\ displaystyle X_ {1}, \ dots, X_ {n}}$

{\ displaystyle D (P_ {0} \ | P_ {1}): = \ int _ {- \ infty} ^ {\ infty} f_ {0} (x) \ cdot \ log {\ frac {f_ {0} (x)} {f_ {1} (x)}} \; \ mathrm {d} x}

the Kullback-Leibler entropy of and ${\ displaystyle P_ {0}}$ ${\ displaystyle P_ {1}}$

statement

Under the above conditions, the following applies: The selectivity of strives with exponential speed towards 1. More precisely, applies ${\ displaystyle \ Phi _ {n}}$

{\ displaystyle \ operatorname {E} _ {P_ {1}} (\ Phi _ {n}) \ approx 1- \ exp (-nD (P_ {0} \ | P_ {1}))}

for big . ${\ displaystyle n}$

interpretation

According to the Neyman-Pearson lemma , Neyman-Pearson tests are consistently the best tests , i.e. they have a greater selectivity than any other test at the same level. Stein's lemma complements this statement by specifying how great the selectivity will be. Thus, Neyman-Pearson tests are not only consistently better than any other test, they are also good in the sense that their selectivity comes arbitrarily close to 1 and that this happens very quickly as the sample size increases.

The decisive factor in the speed of convergence is the Kullback-Leibler entropy. It provides a measure of how well two probability measures can be distinguished from one another on the basis of a sample.

literature

Hans-Otto Georgii : Stochastics . Introduction to probability theory and statistics. 4th edition. Walter de Gruyter, Berlin 2009, ISBN 978-3-11-021526-7 , doi : 10.1515 / 9783110215274 .