Normal distribution model

In statistics , the normal distribution model or Gaussian product model is a special statistical model that is characterized by simple model assumptions. On the one hand, the collection of the data should be stochastically independent of one another; on the other hand, the data should all be normally distributed , depending on the specification with one or two unknown parameters.

The importance of the normal distribution model results both from the fact that it is a very well investigated model for which good parameter estimates, confidence intervals and tests can be given, and from the special position of the normal distribution, which is always set according to the central limit theorem when many, independent, random influences overlap.

Three cases can be distinguished:

• One starts from a known expected value of the normal distributions and tries to make statements about the variance. An example of this would be the calibration of a scale with a specified, standardized weight.
• One starts from a known variance of the normal distributions and tries to make statements about the expected value. This case would arise, for example, in the case of a measurement with a measuring instrument of known inaccuracy that is specified by the manufacturer.
• Both the variance and the expected value are unknown. An example for this case would be the estimation of the shoe size of men: It is not clear which shoe size a man has “on average”, nor is it clear how much the shoe sizes vary.

Different methods are available for each of the three cases.

Expected value known and variance unknown

If the expected value and variance are known, the framework conditions are formalized as follows: The statistical model is given by

${\displaystyle (\mathbb {R} ^{n},{\mathcal {B}}(\mathbb {R} ^{n}),{\mathcal {P}})}$,

where the distribution class is more accurate than

${\displaystyle {\mathcal {P}}=\{{\mathcal {N}}^{\otimes n}(\mu _{0},\sigma ^{2})\;|\;\sigma ^{2}\in (0,\infty )\}}$

is defined. Here is the known expected value. With is the n-fold product measure of probability measure referred. The model is consequently a single-parameter model and a product model . The distribution class is part of the single-parameter exponential family , because the probability density of the normal distribution is represented as ${\displaystyle \mu _{0}}$ ${\displaystyle P^{\otimes n}}$ ${\displaystyle P}$ ${\displaystyle \phi (x)}$

${\displaystyle \phi (x)=\exp(a(\sigma )\cdot b(x)+c(\sigma ))}$with and . ${\displaystyle a(\sigma )=-{\frac {1}{2\sigma ^{2}}},\;b(x)=(x-\mu _{0})^{2}\;}$ ${\displaystyle c(\sigma )=-{\tfrac {1}{2}}\ln(2\pi \sigma )}$

This gives the representation for the probability density over the entire space

${\displaystyle f(x)=\exp \left(-{\frac {1}{2\sigma ^{2}}}\sum _{i=1}^{n}(x_{i}-\mu _{0})^{2}-{\tfrac {n}{2}}\ln(2\pi \sigma ^{2})\right)}$.

The unknown variance is to be estimated, the parameter function to be estimated is thus given by

${\displaystyle g(\sigma ^{2})=\sigma ^{2}}$.

Parameter estimation

Both the maximum likelihood method and the moment method provide the (uncorrected) sample variance as an estimator for the unknown variance

${\displaystyle V(X)={\frac {1}{n}}\sum _{i=1}^{n}(X_{i}-\mu _{0})^{2}}$.

She is expectant . The sufficiency of this estimator follows from the representation of the normal distribution as part of the exponential family and the corresponding canonical statistics . In addition, the estimator is also complete and therefore, according to Lehmann-Scheffé's theorem, a consistently best, unbiased estimator .

Confidence intervals

Confidence intervals for the unknown variance are based on the pivot statistic

${\displaystyle T(X,\sigma ^{2})=\sum _{i=1}^{n}\left({\frac {X_{i}-\mu _{0}}{\sigma }}\right)^{2}={\frac {n}{\sigma ^{2}}}\cdot V(X)}$.

It's chi-square distributed with degrees of freedom , so . A bilateral confidence interval for the confidence level is thus given by ${\displaystyle n}$ ${\displaystyle T(X,\sigma ^{2})\sim \chi _{n}^{2}}$ ${\displaystyle 1-\alpha }$

${\displaystyle C(X)=\left[{\frac {n}{\chi _{n;1-\alpha /2}^{2}}}V(X);{\frac {n}{\chi _{n;\alpha /2}^{2}}}V(X)\right]}$.

Here is the - quantile of the chi-square distribution with degrees of freedom. The concrete values ​​of the quantiles can be looked up in the quantile table of the chi-square distribution . ${\displaystyle \chi _{n,\alpha }^{2}}$ ${\displaystyle \alpha }$ ${\displaystyle n}$

Testing

For one- sample problems there is the chi-square test to check a variance, for two-sample problems the F-test to compare two variances.

Variance known and expected value unknown

If the variance is known and the expected value is unknown, the framework conditions are formalized as follows: the statistical model given by

${\displaystyle (\mathbb {R} ^{n},{\mathcal {B}}(\mathbb {R} ^{n}),{\mathcal {P}})}$,

where the distribution class is more accurate than

${\displaystyle {\mathcal {P}}=\{{\mathcal {N}}^{\otimes n}(\mu ,{\sigma _{0}}^{2})\;|\;\mu \in \mathbb {R} \}}$

is defined. Here denotes the known variance. The model is consequently a single-parameter model and a product model . The distribution class is also a part of the one-parameter exponential family , because the probability density of the normal distribution is represented as ${\displaystyle \sigma _{0}}$ ${\displaystyle \phi (x)}$

${\displaystyle \phi (x)=\exp(a(\mu )\cdot b(x)+c(\mu )+d(x))}$with and . ${\displaystyle a(\mu )={\frac {\mu }{\sigma _{0}^{2}}},\;b(x)=x,\;c(\mu )=-{\tfrac {\mu ^{2}}{2\sigma _{0}^{2}}}}$ ${\displaystyle d(x)=-\left({\tfrac {x^{2}}{2\sigma _{0}^{2}}}+{\tfrac {1}{2}}\ln(2\pi \sigma _{0}^{2})\right)}$

This gives the representation for the probability density over the entire space

${\displaystyle f(x)=\exp \left({\frac {\mu }{\sigma _{0}^{2}}}\sum _{i=1}^{n}x_{i}+n\cdot c(\mu )-\left({\tfrac {n}{2}}\ln(2\pi \sigma _{0}^{2})+\sum _{i=i}^{n}{\tfrac {x_{i}^{2}}{2\sigma _{0}^{2}}}\right)\right)}$

The unknown expected value is to be estimated; the parameter function to be estimated is thus given by

${\displaystyle g(\mu )=\mu }$.

Parameter estimation

Both the maximum likelihood method and the moment method provide the sample mean as an estimator for the expected value

${\displaystyle {\overline {X}}={\frac {1}{n}}\sum _{i=1}^{n}X_{i}}$

the sample. The maximum likelihood estimator follows, for example, by determining the maximum of the log likelihood function , the moment estimator follows directly from the fact that the arithmetic mean is the first empirical moment and the expected value is used to estimate the first stochastic moment shall be.

The estimator is unbiased . Since it is also the canonical statistics of the exponential family, it is also sufficient . In addition, the estimator is also complete and therefore, according to Lehmann-Scheffé's theorem, a consistently best, unbiased estimator .

Confidence intervals

The confidence intervals with known variance are based on the pivot statistic ${\displaystyle \sigma _{0}^{2}}$

${\displaystyle T(X,\mu )={\frac {{\sqrt {n}}({\overline {X}}-\mu )}{\sigma _{0}}}}$.

It is distributed according to the standard normal , i.e. for everyone . ${\displaystyle T(X,\mu )\sim {\mathcal {N}}(0,1)}$ ${\displaystyle \mu }$

Let it denote the - quantile of the standard normal distribution. This can be found in the quantile table of the standard normal distribution. Then a right-hand unlimited confidence interval for the unknown expected value at the confidence level is given by ${\displaystyle u_{\alpha }}$ ${\displaystyle \alpha }$ ${\displaystyle 1-\alpha }$

${\displaystyle C_{1}(X)=\left[{\overline {X}}-{\tfrac {\sigma _{0}}{\sqrt {n}}}u_{1-\alpha };\infty \right)}$.

Similarly, there is an unlimited confidence interval on the left for the unknown expected value for the confidence level through ${\displaystyle 1-\alpha }$

${\displaystyle C_{2}(X)=\left(-\infty ;{\overline {X}}+{\tfrac {\sigma _{0}}{\sqrt {n}}}u_{1-\alpha }\right]}$.

A two-sided confidence interval for the confidence level is given by ${\displaystyle 1-\alpha }$

${\displaystyle C_{3}(X)=\left[{\overline {X}}-{\tfrac {\sigma _{0}}{\sqrt {n}}}u_{1-\alpha /2};{\overline {X}}+{\tfrac {\sigma _{0}}{\sqrt {n}}}u_{1-\alpha /2}\right]}$.

Testing

For -sample problem exists -sample z-test and the sample t-test for two-sample problems of the two-sample z-test .

Variance and expected value unknown

If the expected value and variance are unknown, the framework conditions are formalized as follows: the statistical model is given by

${\displaystyle (\mathbb {R} ^{n},{\mathcal {B}}(\mathbb {R} ^{n}),{\mathcal {P}})}$,

where the distribution class is more accurate than

${\displaystyle {\mathcal {P}}=\{{\mathcal {N}}^{\otimes n}(\mu ,\sigma ^{2})\mid \sigma ^{2}\in (0,\infty ),\mu \in \mathbb {R} \}}$

is defined. This is then a parametric model and a product model . The distribution class is part of the two-parameter exponential family , as it is for the probability density of the normal distribution

${\displaystyle \phi (x)=\exp \left(a_{1}(\mu ,\sigma )\cdot b_{1}(x)+a_{2}(\mu ,\sigma )\cdot b_{1}(x)+c(\mu ,\sigma )\right)}$with and ${\displaystyle a_{1}(\mu ,\sigma )={\tfrac {\mu }{\sigma ^{2}}},a_{2}(\mu ,\sigma )=-{\tfrac {1}{2\sigma ^{2}}},b_{1}(x)=x,b_{2}(x)=x^{2}}$ ${\displaystyle c(\mu ,\sigma )=-{\tfrac {\mu ^{2}}{2\sigma ^{2}}}-{\tfrac {1}{2}}\ln(2\pi \sigma ^{2})}$

applies.

The expected value and variance are to be estimated, the parameter functions to be estimated are thus given by

${\displaystyle g_{1}(\mu ;\sigma ^{2})=\mu }$and . ${\displaystyle g_{2}(\mu ;\sigma ^{2})=\sigma ^{2}}$

Parameter estimation

The maximum likelihood method and the moment method provide the sample mean as an estimate function for the unknown expected value

${\displaystyle {\overline {X}}={\frac {1}{n}}\sum _{i=1}^{n}X_{i}}$.

This estimator is unbiased .

Both the maximum likelihood method and the moment method provide the (uncorrected) sample variance

${\displaystyle V(X)={\tfrac {1}{n}}\sum _{i=1}^{n}(X_{i}-{\overline {X}})^{2}}$

as an estimator for the unknown variance. It is not true to expectation, but only asymptotically true to expectation . The Bessel correction is therefore introduced and the corrected sample variance is obtained as an unbiased estimator

${\displaystyle V^{*}(X)={\tfrac {1}{n-1}}\sum _{i=1}^{n}(X_{i}-{\overline {X}})^{2}}$.

It is an unbiased estimator for the unknown variance.

Confidence intervals

Confidence intervals for the expected value, i.e. for , are based on the pivot statistics in this model ${\displaystyle g(\mu ,\sigma ^{2})=\mu }$

${\displaystyle T(X,\mu )={\sqrt {n}}{\frac {{\overline {X}}-\mu }{S_{n}^{*}(X)}}}$,

in which

${\displaystyle S_{n}^{*}(X)={\sqrt {{\frac {1}{n-1}}\sum _{i=1}^{n}(X_{i}-{\overline {X}})^{2}}}}$

is. This gives you the one-sided confidence interval for the expected value for the confidence level ${\displaystyle 1-\alpha }$

${\displaystyle C_{1}(X)=\left[{\overline {X}}-{\frac {S_{n}^{*}(X)}{\sqrt {n}}}t_{n-1;1-\alpha };\infty \right)}$,

as a two-sided confidence interval for the expected value at the confidence level is obtained ${\displaystyle 1-\alpha }$

${\displaystyle C_{2}(X)=\left[{\overline {X}}-{\frac {S_{n}^{*}}{\sqrt {n}}}t_{n-1;1-\alpha /2};{\overline {X}}-{\frac {S_{n}^{*}}{\sqrt {n}}}t_{n-1;\alpha /2}\right]}$

Here the quantile is the Student's t-distribution with n degrees of freedom. The concrete values ​​of the quantiles can be looked up in the quantile table of Student's t-distribution . ${\displaystyle t_{n,\alpha }}$ ${\displaystyle \alpha }$

Confidence intervals for the variance, i.e. for , are based on the pivot statistic ${\displaystyle g(\mu ,\sigma ^{2})=\sigma ^{2}}$

${\displaystyle T(X,\sigma ^{2})=\left({\frac {S_{n}^{*}}{\sigma }}\right)^{2}}$.

It provides the one-sided confidence interval for the variance to the confidence level ${\displaystyle 1-\alpha }$

${\displaystyle C_{3}(X)=\left[0;{\frac {1}{\chi _{n-1;\alpha }^{2}}}\sum _{i=1}^{n}(X_{i}-{\overline {X}})^{2}\right)}$,

and the two-tailed confidence interval for the variance to the confidence level ${\displaystyle 1-\alpha }$

${\displaystyle C_{4}(X)=\left({\frac {1}{\chi _{n-1;1-\alpha /2}^{2}}}\sum _{i=1}^{n}(X_{i}-{\overline {X}})^{2};{\frac {1}{\chi _{n-1;\alpha /2}^{2}}}\sum _{i=1}^{n}(X_{i}-{\overline {X}})^{2}\right)}$

Here the quantile is the chi-square distribution with degrees of freedom. The concrete values ​​of the quantiles can be looked up in the quantile table of the chi-square distribution . ${\displaystyle \chi _{n,\alpha }^{2}}$ ${\displaystyle \alpha }$ ${\displaystyle n}$

Testing

For one- sample problems there is a chi-square test for the variance to check a variance. For two-sample problems there is an F-test for the variance to compare two variances, for the expected value see Behrens-Fisher problem .

Individual evidence

1. ^ Ludger Rüschendorf: Mathematical Statistics . Springer Verlag, Berlin Heidelberg 2014, ISBN 978-3-642-41996-6 , p. 96 , doi : 10.1007 / 978-3-642-41997-3 .
2. ^ Hans-Otto Georgii: Stochastics . Introduction to probability theory and statistics. 4th edition. Walter de Gruyter, Berlin 2009, ISBN 978-3-11-021526-7 , p. 205 , doi : 10.1515 / 9783110215274 .
3. ^ ^{A b} Ludger Rüschendorf: Mathematical Statistics . Springer Verlag, Berlin Heidelberg 2014, ISBN 978-3-642-41996-6 , p. 110-111 , doi : 10.1007 / 978-3-642-41997-3 .
4. ↑ Claudia Czado, Thorsten Schmidt: Mathematical Statistics . Springer-Verlag, Berlin Heidelberg 2011, ISBN 978-3-642-17260-1 , p. 143-144 , doi : 10.1007 / 978-3-642-17261-8 .
5. ^ ^{A b c} Ludger Rüschendorf: Mathematical Statistics . Springer Verlag, Berlin Heidelberg 2014, ISBN 978-3-642-41996-6 , p. 196 , doi : 10.1007 / 978-3-642-41997-3 .
6. ^ ^{A b c} Ludger Rüschendorf: Mathematical Statistics . Springer Verlag, Berlin Heidelberg 2014, ISBN 978-3-642-41996-6 , p. 231-232 , doi : 10.1007 / 978-3-642-41997-3 .