Standard error

The standard error or sampling error is a measure of dispersion for an estimation function for an unknown parameter of the population . The standard error is defined as the standard deviation of the estimator, i.e. the positive square root of the variance . In the natural sciences and metrology, the term standard uncertainty coined by the GUM is also used. ${\displaystyle {\hat {\vartheta }}}$ ${\displaystyle \vartheta }$ ${\displaystyle \sigma ({\hat {\vartheta }})=+{\sqrt {\operatorname {Var} ({\hat {\vartheta }})}}}$ ${\displaystyle {\hat {\vartheta }}}$

In the case of an unbiased estimator, the standard error is therefore a measure of the average deviation of the estimated parameter value from the true parameter value. The smaller the standard error, the more precisely the unknown parameter can be estimated using the estimator. The standard error depends on, among other things

• the sample size and
• the variance in the population.

In general, the larger the sample size, the smaller the standard error; the smaller the variance, the smaller the standard error.

The standard error also plays an important role in the calculation of estimation errors , confidence intervals and test statistics .

interpretation

The standard error provides information about the quality of the estimated parameter. The more individual values ​​there are, the smaller the standard error and the more accurately the unknown parameter can be estimated. The standard error makes the measured spread (standard deviation) of two data sets with different sample sizes comparable by normalizing the standard deviation to the sample size.

If the unknown parameter is estimated with the help of several samples, the results will vary from sample to sample. Of course, this variation does not come from a variation of the unknown parameter (because it is fixed), but from random influences, e.g. B. Measurement inaccuracies. The standard error is the standard deviation of the estimated parameters in many samples. In general, the following applies: To halve the standard error, you need to quadruple the sample size.

In contrast to this, the standard deviation depicts the actual spread in a population , which is also present with the highest measurement accuracy and an infinite number of individual measurements (e.g. weight distribution, size distribution, monthly income). It shows whether the individual values ​​are close together or whether the data are widely spread.

example

Suppose you study the population of high school children for their intelligence performance. So the unknown parameter is the mean intelligence performance of the children attending high school. If a random sample of the size (i.e. with children) is drawn from this population , then the mean value can be calculated from all measurement results . If, after this sample, another random sample with the same number of children is drawn and the mean value is determined, the two mean values ​​will not exactly match. If one draws a large number of other random samples of the scope , then the spread of all empirically determined mean values ​​around the mean value of the population can be determined. This spread is the standard error. Because the mean of the sample means is the best estimate of the population mean, the standard error is the dispersion of the empirical means around the population mean. It does not show the intelligence distribution of the children, but the accuracy of the calculated mean. ${\displaystyle n}$ ${\displaystyle n}$ ${\displaystyle n}$ ${\displaystyle n}$ ${\displaystyle n}$

notation

Various terms are used for the standard error to distinguish it from the standard deviation of the population and to make it clear that it is the spread of the estimated parameter of samples: ${\displaystyle \sigma }$

• ${\displaystyle \sigma _{n}}$,
• ${\displaystyle \sigma ({\hat {\vartheta }})}$ or
• ${\displaystyle \sigma _{\hat {\vartheta }}}$.

estimate

Since the standard error of the population is included in the standard error, the standard deviation in the population must be estimated using an estimator that is as true to expectation as possible in order to estimate the standard error. ${\displaystyle \sigma }$

Confidence Intervals and Tests

The standard error also plays an important role in confidence intervals and tests . If the estimator is fair to expectations and at least approximately normally distributed ( ), then is ${\displaystyle {\hat {\vartheta }}}$ ${\displaystyle {\mathcal {N}}(\vartheta ,\sigma ^{2}({\hat {\vartheta }}))}$

${\displaystyle {\frac {{\hat {\vartheta }}-\vartheta }{\sigma ({\hat {\vartheta }})}}\approx {\mathcal {N}}(0;1)}$.

On this basis, - confidence intervals can be specified for the unknown parameter : ${\displaystyle (1-\alpha )}$ ${\displaystyle \vartheta }$

${\displaystyle P({\hat {\vartheta }}-z_{1-\alpha /2}\sigma ({\hat {\vartheta }})\leq \vartheta \leq {\hat {\vartheta }}+z_{1-\alpha /2}\sigma ({\hat {\vartheta }}))=1-\alpha }$

or formulate tests, e.g. B. whether the parameter assumes a certain value : ${\displaystyle \vartheta _{0}}$

${\displaystyle H_{0}:\vartheta =\vartheta _{0}}$ vs. ${\displaystyle H_{1}:\vartheta \neq \vartheta _{0}}$

and the test statistic results in:

${\displaystyle V={\frac {{\hat {\vartheta }}-\vartheta _{0}}{\sigma ({\hat {\vartheta }})}}\approx {\mathcal {N}}(0;1)}$.

${\displaystyle z_{1-\alpha /2}}$is the - quantile of the standard normal distribution and is also the critical value for the formulated test. As a rule, it must be estimated from the sample, so that ${\displaystyle (1-\alpha /2)}$ ${\displaystyle \sigma ({\hat {\vartheta }})}$

${\displaystyle V={\frac {{\hat {\vartheta }}-\vartheta _{0}}{{\hat {\sigma }}({\hat {\vartheta }})}}\approx t_{n-1}}$

holds, where is the number of observations. For the t-distribution can be approximated by the standard normal distribution. ${\displaystyle n}$ ${\displaystyle n>30}$

Standard error of the arithmetic mean

The standard error of the arithmetic mean is the same

${\displaystyle \sigma ({\overline {X}})={\frac {\sigma }{\sqrt {n}}}}$,

where denotes the standard deviation of a single measurement. ${\displaystyle \sigma }$

Derivation

The mean of a sample size is defined by ${\displaystyle n}$

${\displaystyle {\overline {x}}={\frac {1}{n}}\sum _{i=1}^{n}x_{i}.}$

Looking at the estimator

${\displaystyle {\overline {X}}={\frac {1}{n}}\sum _{i=1}^{n}X_{i}}$

with independent, identically distributed random variables with finite variance , the standard error is defined as the square root of the variance of . Using the calculation rules for variances and the Bienaymé equation, one calculates : ${\displaystyle X_{1},\ldots ,X_{n}}$ ${\displaystyle \sigma ^{2}}$ ${\displaystyle {\overline {X}}}$

${\displaystyle \sigma ({\overline {X}})^{2}=\operatorname {Var} \left({\overline {X}}\right)=\operatorname {Var} \left({\frac {1}{n}}\sum _{i=1}^{n}X_{i}\right)={\frac {1}{n^{2}}}\operatorname {Var} \left(\sum _{i=1}^{n}X_{i}\right)={\frac {1}{n^{2}}}\sum _{i=1}^{n}\operatorname {Var} \left(X_{i}\right)={\frac {1}{n^{2}}}n\sigma ^{2}={\frac {\sigma ^{2}}{n}}}$

from which the formula for the standard error follows. If it holds, it follows analogously ${\displaystyle \operatorname {Var} (X_{i})=\sigma _{i}^{2}}$

${\displaystyle \sigma ({\overline {X}})^{2}={\frac {1}{n^{2}}}\sum _{i=1}^{n}\sigma _{i}^{2}}$.

Calculation of ${\displaystyle \sigma }$

Assuming a sample distribution, the standard error can be calculated using the variance of the sample distribution:

• in the binomial distribution with parameters ${\displaystyle N,\,p}$

${\displaystyle \sigma _{{\bar {x}},\mathrm {binom} }={\frac {\sqrt {N\cdot p\cdot (1-p)}}{\sqrt {n}}}}$,

• for the exponential distribution with parameter (expected value = standard deviation = ): ${\displaystyle \lambda }$ ${\displaystyle 1/\lambda }$

${\displaystyle \sigma _{{\bar {x}},\mathrm {exp} }={\frac {1}{\lambda {\sqrt {n}}}}}$

• and for the Poisson distribution with parameters (expected value = variance = ): ${\displaystyle \lambda }$ ${\displaystyle \lambda }$

${\displaystyle \sigma _{{\bar {x}},\mathrm {poisson} }={\sqrt {\frac {\lambda }{n}}}}$

Designate

• ${\displaystyle \sigma _{{\bar {x}},\mathrm {binom} },\sigma _{{\bar {x}},\mathrm {exp} },\sigma _{{\bar {x}},\mathrm {poisson} }}$ the standard errors of the respective distributions, and
• ${\displaystyle n}$ the sample size.

If the standard error for the mean is to be estimated, then the variance is estimated with the corrected sample variance . ${\displaystyle \sigma ^{2}}$

example

For the ice cream data, the arithmetic mean, its standard error and the standard deviation for the years 1951, 1952 and 1953 were calculated for the per capita consumption of ice cream (measured in pints ).

year	Average	Standard error of the mean	Standard deviation	Number of observations
1951	0.34680	0.01891	0.05980	10
1952	0.34954	0.01636	0.05899	13
1953	0.39586	0.03064	0.08106	7th

For the years 1951 and 1952 the estimated mean values ​​and standard deviations as well as the observation numbers are roughly the same. Therefore, the estimated standard errors also give about the same value. In 1953, on the one hand, the number of observations is lower and the standard deviation is higher. Therefore, the standard error is almost twice the standard errors from 1951 and 1952.

The graphical representation can take place by means of an error bar diagram. The 95% estimate intervals for the years 1951, 1952 and 1953 are shown on the right. If the sample function is at least approximately normally distributed, then the 95% estimation intervals are given by with and the sample means and the sample variances. ${\displaystyle {\bar {X}}}$ ${\displaystyle {\bar {x_{j}}}\pm 1{,}96\cdot s_{j}/{\sqrt {n_{j}}}}$ ${\displaystyle j=1951,1952,1953}$ ${\displaystyle {\bar {x}}_{j}}$ ${\displaystyle s_{j}^{2}}$

Here, too, one can clearly see that the mean value for 1953 can be estimated more imprecisely than the mean values ​​for 1951 and 1952 (longer bar for 1953).

Standard error of the regression coefficients in the simple regression model

The classic regression model for simple linear regression assumes that ${\displaystyle Y_{i}=\beta _{0}+\beta _{1}x_{i}+\varepsilon _{i}}$

• the disturbance terms are normally distributed , ${\displaystyle \varepsilon _{i}\sim \,(0,\sigma ^{2})}$
• the terms are independent and
• the values ​​are fixed (i.e. no random variables), ${\displaystyle x_{i}}$

with the observations made by running. For the estimators ${\displaystyle i=1,\ldots ,n}$

${\displaystyle {\hat {\beta }}_{1}={\frac {\sum _{i}(x_{i}-{\overline {x}})(Y_{i}-{\overline {Y}})}{\sum _{i}(x_{i}-{\overline {x}})^{2}}}}$ and

${\displaystyle {\hat {\beta }}_{0}={\overline {Y}}-{\hat {\beta }}_{1}{\overline {x}}}$

then results

${\displaystyle {\hat {\beta }}_{1}\sim {\mathcal {N}}(\beta _{1},\sigma _{{\hat {\beta }}_{1}}^{2}\cdot a_{1})\,}$and . ${\displaystyle {\hat {\beta }}_{0}\sim {\mathcal {N}}(\beta _{0},\sigma _{{\hat {\beta }}_{0}}^{2}\cdot a_{0})}$

The standard errors of the regression coefficients are given by

${\displaystyle \sigma _{{\hat {\beta }}_{1}}=\operatorname {SD} ({\hat {\beta }}_{1})=\sigma {\sqrt {\underbrace {\frac {1}{\sum \nolimits _{i=1}^{n}(x_{i}-{\overline {x}})^{2}}} _{=:a_{1}}}}=\sigma \cdot {\sqrt {a_{1}}}}$

and

${\displaystyle \sigma _{{\hat {\beta }}_{0}}=\operatorname {SD} ({\hat {\beta }}_{0})=\sigma {\sqrt {\underbrace {\frac {\sum \nolimits _{i=1}^{n}x_{i}^{2}}{n\sum \nolimits _{i=1}^{n}(x_{i}-{\overline {x}})^{2}}} _{=:a_{0}}}}=\sigma \cdot {\sqrt {a_{0}}}}$.

Example : For the ice cream data, a simple linear regression was carried out for the per capita consumption of ice cream (measured in half liters) with the mean weekly temperature (in Fahrenheit) as an independent variable. The estimation of the regression model resulted in:

${\displaystyle {\text{Pro-Kopf-Verbrauch}}=0{,}20686+0{,}00311\cdot {\text{Temperatur}}}$.

model	Non-standardized coefficients		Standardized coefficients	T	Sig.
	Regression coefficients	Standard error
constant	0.20686	0.02470		8.375	0.000
temperature	0.00311	0.00048	0.776	6.502	0.000

Although the estimated regression coefficient for the mean weekly temperature is very small, the estimated standard error gave an even smaller value. The accuracy with which the regression coefficient is estimated is a good 6.5 times as small as the coefficient itself.

See also

• Standard error of regression

Individual evidence

1. ↑ ^{a b} Koteswara Rao Kadiyala (1970): Testing for the independence of regression disturbances. In: Econometrica , 38, 97-117.
2. ↑ ^{a b} ice cream dates. In: Data and Story Library , accessed February 16, 2010