Linear predictor

In statistics, and particularly in parametric regression analysis , a linear predictor is a linear combination of a series of coefficients ( regression coefficients ) and explanatory variables ( independent variables ), the value of which is used to predict a response variable. This additive-linear systematic component is a main part of linear regression models .

definition

In parametric regression analysis, a search space is formed from pontential regression functions using several regression parameters. Then the parameter configuration should be determined that provides the highest quality of fit  for the observed values ​​of the response variables and explanatory variables. The most important model classes of parametric regression analysis are, on the one hand, the class of linear models and, on the other hand, the class of generalized linear models . The adjective “linear” results from the fact that the two model classes are based on the linear predictor , which is defined as follows

${\displaystyle \eta _{i}\colon =x_{i0}\beta _{0}+x_{i1}\beta _{1}+x_{i2}\beta _{2}+\ldots +x_{ik}\beta _{k}=\sum \nolimits _{j=0}^{k}x_{ij}\beta _{j}}$.

This linear predictor is made up of the explanatory variables and the fixed but unknown regression parameters, usually equal to one ( ). The parameter is thus the intercept of the regression line or, more precisely, "regression hyperplane ". It determines the level of the linear predictor and is therefore also called the level parameter. The regression analysis is about estimating the axis intercept , the slope parameters and the variance of the disturbance variables. ${\displaystyle x_{i0},x_{i1},\ldots ,x_{ik}}$ ${\displaystyle \beta _{0},\beta _{1},\beta _{2},\ldots ,\beta _{k}}$ ${\displaystyle x_{i0}}$ ${\displaystyle x_{i0}\equiv 1}$ ${\displaystyle \beta _{0}}$ ${\displaystyle \beta _{0}}$ ${\displaystyle \beta _{1},\beta _{2},\ldots ,\beta _{k}}$

Linear models vs. generalized linear models

Linear models assume the following relationship between the regression function and the linear predictor

${\displaystyle f(x_{i1},x_{i2},\ldots ,x_{ik})=\sum \nolimits _{j=0}^{k}x_{ij}\beta _{j}=\eta _{i}}$.

Generalized linear models, on the other hand, assume that the expected value of the response variable only takes the form of a linear predictor through a suitable invertible coupling function ${\displaystyle \mu =\operatorname {E} (Y_{i})}$ ${\displaystyle g(\cdot )}$

${\displaystyle g(\mu )=\sum \nolimits _{j=0}^{k}x_{ij}\beta _{j}=\eta _{i}}$.

With the inverse function of the coupling function , the response function results for the regression function in this case ${\displaystyle h(\cdot )=g^{-1}(\cdot )}$

${\displaystyle f(x_{i1},x_{i2},\ldots ,x_{ik})=h\left(\sum \nolimits _{j=0}^{k}x_{ij}\beta _{j}\right)=h(\eta _{i})}$.

In additive models and generalized additive models with the observations of a continuous response variable and the explanatory variables , continuous explanatory variables are also added. The linear predictor is then given by: ${\displaystyle (y_{i};x_{i1},x_{i2},\ldots ,x_{ik}),\,i=1,\ldots ,n}$ ${\displaystyle Y_{i}}$ ${\displaystyle x_{i1},x_{i2},\ldots ,x_{ik}}$ ${\displaystyle (z_{i1},z_{i2},\ldots ,z_{iq}),\,i=1,\ldots ,n}$

${\displaystyle \eta _{i}^{add.}=f_{1}(z_{i1})+\ldots +f_{q}(z_{iq})+\eta _{i}}$

The functions here represent the non-linear smoothing effects of the explanatory variables and are modeled and estimated using non-parametric methods. ${\displaystyle f_{1}(z_{1}),\ldots ,f_{q}(z_{q})}$

Vector matrix notation

In vector matrix notation, the linear predictor can be written as follows:

${\displaystyle \eta _{i}=\beta _{0}+x_{i1}\beta _{1}+x_{i2}\beta _{2}+\dotsc +x_{ik}\beta _{k}=\mathbf {x} _{i}^{\top }{\boldsymbol {\beta }}\quad }$, where and ${\displaystyle \quad \mathbf {x} _{i}^{\top }=(1,x_{i1},\ldots ,x_{ik})_{(k+1\times 1)}\quad }$ ${\displaystyle \quad {\boldsymbol {\beta }}=(\beta _{0},\beta _{1},\ldots ,\beta _{k})_{(k+1\times 1)}^{\top }}$

here is a -column vector and is a transposed -column vector, so that the product results in a matrix or a scalar. ${\displaystyle {\boldsymbol {\beta }}}$ ${\displaystyle (k+1)\times 1}$ ${\displaystyle \mathbf {x} _{i}^{\top }}$ ${\displaystyle (k+1)\times 1}$ ${\displaystyle \mathbf {x} _{i}^{\top }{\boldsymbol {\beta }}}$ ${\displaystyle 1\times 1}$

Use in linear regression

→ Main article : Linear regression

An example of using a linear predictor is linear regression , in which an additive confounding variable is superimposed on each of the relationship between explanatory variables and response variables. In multiple linear regression , the relationship can be written as follows:

${\displaystyle Y_{i}=\eta _{i}+\varepsilon _{i}=\mathbf {x} _{i}^{\top }{\boldsymbol {\beta }}+\varepsilon _{i}}$.

Individual evidence

1. ^ Ludwig Fahrmeir, Thomas Kneib , Stefan Lang, Brian Marx: Regression: models, methods and applications. Springer Science & Business Media, 2013, ISBN 978-3-642-34332-2 , p. 536.