Lemma of Stein
The statement is named after Charles Stein , who proved it in 1952.
Let a statistical model be given with a simple null hypothesis and a simple alternative , for which both the probability density functions and exist and are genuinely positive. Furthermore, let the corresponding infinite product model and the projection onto the components of the product model.
Be a Neyman-Pearson test at the level that only depends on. Furthermore denote
the Kullback-Leibler entropy of and
Under the above conditions, the following applies: The selectivity of strives with exponential speed towards 1. More precisely, applies
for big .
According to the Neyman-Pearson lemma , Neyman-Pearson tests are consistently the best tests , i.e. they have a greater selectivity than any other test at the same level. Stein's lemma complements this statement by specifying how great the selectivity will be. Thus, Neyman-Pearson tests are not only consistently better than any other test, they are also good in the sense that their selectivity comes arbitrarily close to 1 and that this happens very quickly as the sample size increases.
The decisive factor in the speed of convergence is the Kullback-Leibler entropy. It provides a measure of how well two probability measures can be distinguished from one another on the basis of a sample.