Standard error of the regression coefficient

In statistics , the standard error of the regression coefficient is a measure of the variability of the estimator for the regression coefficient . The standard error of the regression coefficient is required to be able to assess the precision of the estimate of the regression coefficient, for example using a statistical test or a confidence interval .

The estimation of a regression line is often given in such a way that the standard errors are given in brackets under the coefficients determined. This provides a clear representation of the two aspects, the point estimate and an indication of the precision.

Special case: linear single regression

In the linear single regression (only one explanatory variable) the (estimated) standard error of the regression coefficient (here the increase ) is given by the positive square root of the estimated variance of (see also Linear single regression # variances of the least squares estimator ): ${\displaystyle y_{i}=\beta _{0}+x_{i}\beta _{1}+\varepsilon _{i}}$ ${\displaystyle {\hat {\beta }}_{1}}$ ${\displaystyle {\hat {\beta }}_{1}}$

${\displaystyle {\hat {\sigma }}_{{\hat {\beta }}_{1}}=\operatorname {SE} ({\hat {\beta }}_{1})={\sqrt {\frac {{\hat {\sigma }}^{2}}{\sum \nolimits _{i=1}^{n}(x_{i}-{\overline {x}})^{2}}}}}$

with the unbiased estimation of the variance of the disturbance variables

${\displaystyle {\hat {\sigma }}^{2}={\frac {1}{n-2}}\sum \limits _{i=1}^{n}(y_{i}-{\hat {\beta }}_{0}-{\hat {\beta }}_{1}x_{i})^{2}}$,

its empirical counterpart is the mean residual square

${\displaystyle MQR={\sqrt {\frac {SQR}{n-2}}}}$

is. Hence for the (empirical) standard error of the slope:

${\displaystyle s_{b_{1}}={\sqrt {\frac {MQR}{\sum \nolimits _{i=1}^{n}(x_{i}-{\overline {x}})^{2}}}}}$.

This standard error can be used to assess the precision of the estimate of the slope , e.g. B. by means of a confidence interval . A confidence interval for the unknown (true) regression coefficient is given by: ${\displaystyle \beta _{1}}$ ${\displaystyle (1-\alpha )}$ ${\displaystyle \beta _{1}}$

${\displaystyle KI_{1-\alpha }(\beta _{1})=\left[b_{1}-s_{b_{1}}t_{1-\alpha /2}(n-2);b_{1}+s_{b_{1}}t_{1-\alpha /2}(n-2)\right]}$,

wherein the - quantile of the Student's t distribution with degrees of freedom is. ${\displaystyle t_{1-\alpha /2}(n-2)}$ ${\displaystyle (1-\alpha /2)}$ ${\displaystyle (n-2)}$

Multiple linear regression

Given a typical multiple linear regression model , with the vector of the unknown regression parameters , the experiment plan matrix , the vector of the dependent variables and the vector of the disturbance variables . Then, in multiple linear regression, the unbiased estimate of the variance of the disturbance variables in matrix notation is given by the sum of squares adjusted by the residual degrees of freedom: ${\displaystyle \mathbf {y} =\mathbf {X} {\boldsymbol {\beta }}+{\boldsymbol {\varepsilon }}}$ ${\displaystyle {\boldsymbol {\beta }}}$ ${\displaystyle p\times 1}$ ${\displaystyle n\times p}$ ${\displaystyle \mathbf {X} }$ ${\displaystyle n\times 1}$ ${\displaystyle \mathbf {y} }$ ${\displaystyle n\times 1}$ ${\displaystyle {\boldsymbol {\varepsilon }}}$

${\displaystyle {\hat {\sigma }}^{2}=SQR/(n-k-1)={\frac {{\hat {\boldsymbol {\varepsilon }}}^{\top }{\hat {\boldsymbol {\varepsilon }}}}{n-k-1}}}$.

For the standard error of the (estimated) regression coefficients it follows that they are given by the square root of the -th diagonal element of the estimated covariance matrix of the least squares estimator ${\displaystyle j}$

${\displaystyle \operatorname {SE} ({\hat {\beta }}_{j})={\sqrt {{\hat {\sigma }}^{2}(\mathbf {X} ^{\top }\mathbf {X} )_{jj}^{-1}}}}$.

Another representation using the coefficient of determination is

${\displaystyle \operatorname {SE} ({\hat {\beta }}_{j})={\sqrt {\frac {{\tfrac {1}{n-p}}\sum \nolimits _{i=1}^{n}{\hat {\varepsilon }}_{i}^{2}}{(1-{\mathit {R}}_{j}^{2})\sum \nolimits _{i=1}^{n}(x_{ij}-{\overline {x}}_{j})^{2}}}}}$.

Individual evidence

1. ↑ Axel M. Gressner, Torsten Arndt: Lexicon of Medical Laboratory Diagnostics . Springer-Verlag, 2013., 3rd edition, p. 2209.
2. ↑ Werner Timischl : Applied Statistics. An introduction for biologists and medical professionals. 2013, 3rd edition, p. 313.
3. ↑ Werner Timischl : Applied Statistics. An introduction for biologists and medical professionals. 2013, 3rd edition, p. 327.
4. ↑ Lothar Sachs , Jürgen Hedderich: Applied Statistics: Collection of Methods with R. 8., revised. and additional edition. Springer Spectrum, Berlin / Heidelberg 2018, ISBN 978-3-662-56657-2 , p. 806
5. ↑ Jeffrey Marc Wooldridge: Introductory econometrics: A modern approach. 5th edition. Nelson Education, 2015, p. 101.