Generalized linear models

This article deals with a model class that allows the target quantity to assume a distribution other than the normal distribution.

• For the generalized least squares estimation model, see The Generalized Linear Regression Model (VLR) .

Generalized linear models ( VLM ), also generalized linear models ( GLM or GLiM ), are an important class of non-linear models introduced by John Nelder and Robert Wedderburn (1972) in statistics , which represent a generalization of the classic linear regression model in regression analysis . While one assumes in classical linear models that the disturbance variable (the unobservable random component) is normally distributed , in GLMs it can have a distribution from the class of the exponential family . In addition to the normal distribution , this distribution class also includes the binomial , Poisson , Gamma and inverse Gaussian distributions . The use of the exponential family in generalized linear models thus offers a uniform framework for these distributions. The large class of vektorverallgemeinerten linear models ( English vector generalized linear models , short VGLMs ) includes the class of generalized linear models as a special case. Also included in this large class of models are loglinear models for categorical data and the Poisson regression model for count data . In order to overcome the limitations of the generalized linear models and generalized additive models , so-called generalized additive models were developed for position, scale and shape parameters .

Disambiguation

Generalized linear models are not to be confused with the general linear model , whose natural English abbreviation is also GLM , but, in contrast to generalized linear models, assumes a normally distributed response variable. In many statistical program packages - since the abbreviation GLM is already used for the general linear model - other abbreviations such as VLM or GLZ for English GeneraLiZed linear models (in STATISTICA ) or GzLM for English GeneraLiZed Linear Models (in SPSS ) are used to better distinguish them . Some authors use the abbreviation GLiM instead of the abbreviation GLM for better differentiation .

Likewise, generalized linear models are not to be confused with the generalized linear regression model of the generalized least squares estimation ( VKQ estimation ), which however has a generalized structure with regard to the disturbance variables .

Model components

The model class of generalized linear models consists of three components:

• Random component: As with the classical linear models , one assumes independent random variables with expected value that have a density function from the exponential family (e.g. a binomial , Poisson , or gamma distribution ). ${\displaystyle Y_{1},Y_{2},\ldots ,Y_{n}}$ ${\displaystyle \operatorname {E} (Y_{i})=\mu _{i}}$

• Systematic component : The covariate vector is given (see The classical model of linear multiple regression ), which only influences the distribution of the by a linear function . This linear function is called the linear predictor and is given in multiple linear regression in the following form: ${\displaystyle \mathbf {x} _{i}^{\top }=(1,x_{i1},\ldots ,x_{ik})_{(k+1\times 1)}}$ ${\displaystyle Y_{i}}$

${\displaystyle \eta _{i}=\beta _{0}+x_{i1}\beta _{1}+x_{i2}\beta _{2}+\dotsc +x_{ik}\beta _{k}=\mathbf {x} _{i}^{\top }{\boldsymbol {\beta }}}$. Here you can see that the linear predictor introduces the vector of the regression coefficients into the model. ${\displaystyle {\boldsymbol {\beta }}=\left(\beta _{0}\,\beta _{1},\dots ,\beta _{k}\right)^{\top }}$

• Coupling function : For a generalized linear model, a (often non-linear) is coupling function exist, the passing through the linear predictor systematic component and described by the expected value of the response variable stochastic component of the described distribution of coupled: . The inverse function of the coupling function, called the response function transforms the linear combination of the explanatory variables in the (conditional) expected value : . ${\displaystyle g(\cdot )}$ ${\displaystyle \eta _{i}}$ ${\displaystyle \mu =\operatorname {E} (Y_{i})}$ ${\displaystyle Y_{i}}$ ${\displaystyle g(\mu )=\eta _{i}}$ ${\displaystyle h(\cdot )}$ ${\displaystyle \mu =\operatorname {E} (Y_{i})}$ ${\displaystyle \mu _{i}=h(\eta _{i})}$

Distributions from the family of generalized linear models

The normal distribution , binomial distribution , Poisson distribution , gamma distribution and the inverse normal distribution , Bernoulli distribution , scaled Poisson distribution , scaled binomial distribution , scaled negative binomial distribution can be embedded in the model class of the generalized linear models .

Exponential family

The distribution of a response variable belongs to the one-dimensional exponential family if the density function or probability function can be written in the following form: ${\displaystyle Y_{i}}$

${\displaystyle f(y_{i}\mid \theta _{i})=\exp \left({\frac {y_{i}\theta _{i}-b(\theta _{i})}{\phi }}\cdot w_{i}+c(y_{i},\phi ,w_{i})\right)}$.

Here are:

• ${\displaystyle y_{i}}$the observation values ​​of the response variable (known)
• ${\displaystyle w_{i}}$ the specified weights (known)
• ${\displaystyle b(\theta _{i})}$a pre - specified twofold differentiable function (known)
• ${\displaystyle \theta _{i}}$the real-valued distribution parameter of density ; the so-called canonical (natural) parameter (unknown)
• ${\displaystyle \phi }$a scale parameter that is independent of the expected value (also known as the variance parameter ) and that is relevant for the variance (known)
• and a suitable function for normalizing the density ( normalization constant ) and which does not depend on (known) ${\displaystyle c(y_{i},\phi ,w_{i})}$ ${\displaystyle \theta _{i}}$

For the function it is necessary that it can be normalized and that the first and second derivative exist. The second derivative determined in addition to the scale parameter , the variance of the distribution, and therefore as a variance function referred to. For all distributions of the exponential family: ${\displaystyle b(\theta _{i})}$ ${\displaystyle f(y_{i}\mid \theta _{i})}$ ${\displaystyle b^{\prime }(\theta _{i})={\frac {\mathrm {d} \,b(\theta _{i})}{\mathrm {d} \,\theta _{i}}}}$ ${\displaystyle b^{\prime \prime }(\theta _{i})={\frac {\mathrm {d} ^{2}\,b(\theta _{i})}{\mathrm {d} \,\theta _{i}^{2}}}}$ ${\displaystyle b^{\prime \prime }(\theta _{i})}$ ${\displaystyle \phi }$

1. ${\displaystyle \operatorname {E} (Y_{i})=\mu =b^{\prime }(\theta _{i})}$
2. ${\displaystyle \operatorname {Var} (Y_{i})=\sigma ^{2}=\phi \cdot b^{\prime \prime }(\theta _{i})/w_{i}}$

The parameter is not primarily of interest and is therefore regarded as a disturbance parameter . Examples of distributions belonging to the exponential family: ${\displaystyle \phi }$

distribution ${\displaystyle \operatorname {E} (Y_{i})=\mu }$	Canonical parameter ${\displaystyle \theta _{i}}$	Scale parameters ${\displaystyle \phi }$	pre-specified function ${\displaystyle a(\phi )}$	pre-specified function ${\displaystyle b(\theta _{i})}$	Normalization constant ${\displaystyle c(y_{i},\phi ,w_{i})}$	Probability function ${\displaystyle f(y_{i})}$
Normal distribution	${\displaystyle \mu }$	${\displaystyle \sigma ^{2}}$	${\displaystyle \phi }$	${\displaystyle {\frac {\theta _{i}^{2}}{2}}}$	${\displaystyle {\frac {-y_{i}^{2}}{2\phi }}-\log \left({\sqrt {2\pi \phi }}\right)}$	${\displaystyle {\frac {1}{\sqrt {2\pi \sigma ^{2}}}}\exp \left(-{\frac {(y_{i}-\mu )^{2}}{2\sigma ^{2}}}\right)}$
Bernoulli distribution	${\displaystyle \log \left({\frac {\mu }{1-\mu }}\right)}$	${\displaystyle -}$	${\displaystyle 1}$	${\displaystyle \log(1+e^{\theta _{i}})}$	${\displaystyle 0}$	${\displaystyle \mu ^{y_{i}}(1-\mu )^{1-y_{i}}\,}$ With ${\displaystyle y_{i}=0{\text{ oder }}1}$
Binomial distribution	${\displaystyle \log \left({\frac {\mu }{n-\mu }}\right)}$	${\displaystyle -}$	${\displaystyle 1}$	${\displaystyle n\log(1+e^{\theta _{i}})}$	${\displaystyle \log {\binom {n}{y_{i}}}}$	${\displaystyle {\binom {n}{y_{i}}}\left({\frac {\mu }{n}}\right)^{y_{i}}\left(1-{\frac {\mu }{n}}\right)^{n-y_{i}}\;}$ With ${\displaystyle \;y=0,1,\ldots ,n}$
Poisson distribution	${\displaystyle \log(\mu )}$	${\displaystyle -}$	${\displaystyle 1}$	${\displaystyle \exp(\theta _{i})}$	${\displaystyle -\log(y_{i}!)}$	${\displaystyle {\frac {\mu ^{y_{i}}}{y_{i}!}}\exp(-\mu )}$ With ${\displaystyle y_{i}=0,1,\ldots }$

literature

• John Nelder, Peter McCullagh: Generalized Linear Models , Chapman and Hall / CRC Press, 2nd edition 1989

Individual evidence

1. ↑ generalized linear model. Glossary of statistical terms. In: International Statistical Institute . June 1, 2011, accessed July 4, 2020 . 
2. ^ John Nelder, Robert Wedderburn: Generalized Linear Models . In: Journal of the Royal Statistical Society, Series A (General) . 135, 1972, pp. 370-384. doi : 10.2307 / 2344614 .
3. ^ Rencher, Alvin C., and G. Bruce Schaalje: Linear models in statistics. , John Wiley & Sons, 2008., p. 513.
4. ^ Rencher, Alvin C., and G. Bruce Schaalje: Linear models in statistics. , John Wiley & Sons, 2008., p. 514.
5. ^ Ludwig Fahrmeir, Thomas Kneib, Stefan Lang, Brian Marx: Regression: models, methods and applications. Springer Science & Business Media, 2013, ISBN 978-3-642-34332-2 , p. 301.
6. ↑ Torsten Becker, et al .: Stochastic risk modeling and statistical methods. Springer Spectrum, 2016. p. 308.
7. ^ Ludwig Fahrmeir, Thomas Kneib, Stefan Lang, Brian Marx: Regression: models, methods and applications. Springer Science & Business Media, 2013, ISBN 978-3-642-34332-2 , p. 301.
8. ^ Ludwig Fahrmeir , Thomas Kneib , Stefan Lang, Brian Marx: Regression: models, methods and applications. Springer Science & Business Media, 2013, ISBN 978-3-642-34332-2 , p. 302.