Neyman-Pearson lemma

The Neyman-Pearson lemma , also called the Neyman-Pearson Fundamentalsemma or the Fundamentalsallemma of mathematical statistics , is a central set of test theory and thus also of mathematical statistics , which makes an optimality statement about the construction of a hypothesis test . The subject of the Neyman-Pearson lemma is the simplest conceivable scenario of a hypothesis test, which is also called the Neyman-Pearson test : Both the null hypothesis and the alternative hypothesis are simple ; That is, they each correspond to a single probability distribution , the associated probability densities of which are referred to below as and . Then, according to the statement of the Neyman-Pearson lemma, the best test is obtained by a decision in which the null hypothesis is rejected if the likelihood quotient falls below a certain value. ${\ displaystyle H_ {0}}$ ${\ displaystyle H_ {1}}$ ${\ displaystyle f_ {0}}$ ${\ displaystyle f_ {1}}$ ${\ displaystyle f_ {0} / f_ {1}}$

The lemma is named after Jerzy Neyman and Egon Pearson , who proved it in 1933.

situation

We are looking for a hypothesis test that is as "good" as possible and that should lead to a decision between null and alternative hypotheses with high reliability. It is assumed that the null and alternative hypotheses each correspond to exactly one probability distribution that applies to the observation results. Under this prerequisite, the probability of a wrong test decision can be calculated exactly for each definition of a rejection area: In detail, these are the two probabilities for an error of the first type and an error of the second type . Therefore, given an upper limit for an error of the first type given by the significance level, the theoretically conceivable test decisions can be compared with one another in a particularly simple manner in qualitative terms.

Formal description of the situation

Realizations of a real random vector with dimension with values in the measurement space are observed . The exact distribution of . The null hypothesis “ ” is to be tested against the alternative “ ” for two probability measures over the given measurement range. The dimensions and have densities or with respect to the Lebesgue measure , i.e. i.e., they are continuous distributions based on . ${\ displaystyle X}$ ${\ displaystyle d}$ ${\ displaystyle (\ mathbb {R} ^ {d}, {\ mathcal {B}} ^ {d})}$ ${\ displaystyle P_ {X}}$ ${\ displaystyle X}$ ${\ displaystyle P_ {X} = P_ {0}}$ ${\ displaystyle P_ {X} = P_ {1}}$ ${\ displaystyle P_ {0}, P_ {1}}$ ${\ displaystyle P_ {0}}$ ${\ displaystyle P_ {1}}$ ${\ displaystyle f_ {0}}$ ${\ displaystyle f_ {1}}$ ${\ displaystyle \ mathbb {R} ^ {d}}$

A decision-making process is now characterized by the definition of a rejection area , with the help of which the null hypothesis is rejected precisely when the observed realization of in lies. This test must not exceed a specified level , ${\ displaystyle B \ in {\ mathcal {B}} ^ {d}}$ ${\ displaystyle X}$ ${\ displaystyle B}$ ${\ displaystyle \ alpha \ in (0,1)}$

{\ displaystyle P_ {0} (B) = \ int 1_ {B} (x) f_ {0} (x) dx \ leq \ alpha}

,

d. In other words, the probability of a false rejection of the null hypothesis, the so-called error of type I , must not be greater than . Of all tests that adhere to this level, the one called the strongest test that maximizes the so-called test strength , i.e. a minimal error of the 2nd type , ${\ displaystyle \ alpha}$ ${\ displaystyle P_ {1} (B)}$

{\ displaystyle P_ {1} (B ^ {\ complement}) = \ int 1_ {B ^ {\ complement}} (x) f_ {1} (x) dx}

,

the probability that the null hypothesis will not be rejected incorrectly.

formulation

The Neyman-Pearson lemma

In the above situation one considers the extended likelihood quotient for a realization ${\ displaystyle X}$

{\ displaystyle q (x): = {\ begin {cases} {\ frac {f_ {0} (x)} {f_ {1} (x)}} \, & f_ {1} (x)> 0 \\ 1 \, & f_ {0} (x) = f_ {1} (x) = 0 \\\ infty \, & f_ {0} (x)> 0, f_ {1} (x) = 0 \ end {cases} } \.}

The case is only defined for the sake of completeness, since it does not occur with any positive probability. ${\ displaystyle f_ {0} (x) = f_ {1} (x) = 0}$

Now a test of the hypothesis “ ” against the alternative “ ” is optimal at a given level (strongest test) if and only if one exists, so that its rejection area meets the requirements ${\ displaystyle P_ {X} = P_ {0}}$ ${\ displaystyle P_ {X} = P_ {1}}$ ${\ displaystyle \ alpha \ in (0,1)}$ ${\ displaystyle \ gamma \ in (0, \ infty)}$ ${\ displaystyle B \ in {\ mathcal {B}} ^ {d}}$

${\ displaystyle P_ {0} (B) = \ alpha}$ such as
${\ displaystyle q (x) \ leq \ gamma}$ for almost certainly everyone and ${\ displaystyle x \ in B}$
${\ displaystyle q (x) \ geq \ gamma}$ for almost certainly everyone ${\ displaystyle x \ in B ^ {\ complement}}$

Fulfills. The almost certain properties from 2. and 3. relate to the probability measure , i.e. H. they must almost certainly with respect to and enter. ${\ displaystyle 0 {,} 5 (P_ {0} + P_ {1})}$ ${\ displaystyle P_ {0}}$ ${\ displaystyle P_ {1}}$

If a fault area fulfills requirements 1–3., It is also called a Neyman-Pearson area . In discrete models, such a rejection area only exists at certain levels ; in order to fully exploit a given level, randomized tests may have to be used. ${\ displaystyle B}$ ${\ displaystyle \ alpha}$

special cases

At least the following special cases were not considered by the above lemma:

The fault area is the strongest test at the test level ; H. the test has no Type I error. The corresponding test parameter is . ${\ displaystyle B_ {0} = \ {f_ {0} = 0 \}}$ ${\ displaystyle \ alpha = 0}$ ${\ displaystyle \ gamma = 0}$
The fault area is the strongest test of the level because it has the test strength , i. H. the test has no type 2 error. The corresponding test parameter is . ${\ displaystyle B_ {1} = \ {f_ {1}> 0 \}}$ ${\ displaystyle \ alpha = P_ {0} (B_ {1})}$ ${\ displaystyle P_ {1} (B_ {1}) = 1}$ ${\ displaystyle \ gamma = \ infty}$

literature

Jörg Bewersdorff : Statistics - how and why they work. A math reader . Vieweg + Teubner Verlag, 2011., ISBN 978-3834817532 , pp. 196-201, doi : 10.1007 / 978-3-8348-8264-6 .
Edward J. Dudewicz, Satya N. Mishra: Modern Mathematical Statistics . John Wiley & Sons., 1988 ..
Jerzy Neyman, Egon Pearson: On the Problem of the Most Efficient Tests of Statistical Hypotheses . In: Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character . 231, 1933, pp. 289-337. doi : 10.1098 / rsta.1933.0009 .

Individual evidence

^ Neyman-Pearson Lemma . In: Michiel Hazewinkel (Ed.): Encyclopaedia of Mathematics . Springer-Verlag , Berlin 2002, ISBN 978-1-55608-010-4 (English, online ).

Web links

cnx.org: Neyman-Pearson criterion

[1] Neyman-Pearson Lemma . In: Michiel Hazewinkel (Ed.): Encyclopaedia of Mathematics . Springer-Verlag , Berlin 2002, ISBN 978-1-55608-010-4 (English, online ).