Linear model

In statistics , the term linear model ( LM for short ) is used in different ways and in different contexts. The term occurs most frequently in regression analysis and is mostly used synonymously with the term linear regression model . However, the term is also used in time series analysis , where it has a different meaning. In any case, the attribution “linear” is used to refer to a specific class of models.

Linear regression models

In the case of linear regression, a linear model is defined as follows: Let the random sample be given with the realizations . The relationship between the dependent variables and the independent variables is formulated as follows: ${\ displaystyle (Y_ {i}; X_ {i1}, \ ldots, X_ {ip}), \, i = 1, \ ldots, n}$ ${\ displaystyle X_ {1} = x_ {1}, \ ldots, X_ {n} = x_ {n}}$ ${\ displaystyle Y}$ ${\ displaystyle x_ {1}, \ ldots x_ {n}}$

{\ displaystyle Y_ {i} = \ beta _ {0} + \ beta _ {1} \ phi _ {1} (x_ {i1}) + \ ldots + \ beta _ {p} \ phi _ {p} ( x_ {ip}) + \ varepsilon _ {i} \ quad, i = 1, \ ldots, n}

,

where can represent non-linear functions. In the above regression equation, the disturbance terms represent random variables. The epithet results from the requirement that the regression equation is linear in the regression parameters . For example would not be allowed. As an alternative to the above equation, one can also say that the predicted values of the dependent variables are given by the following equation: ${\ displaystyle \ phi _ {1}, \ ldots, \ phi _ {p}}$ ${\ displaystyle \ varepsilon _ {i}}$ ${\ displaystyle \ beta _ {j}}$ ${\ displaystyle \ beta _ {j} ^ {2}}$

{\ displaystyle {\ widehat {Y}} _ {i} = b_ {0} + b_ {1} \ phi _ {1} (x_ {i1}) + \ ldots + b_ {p} \ phi _ {p} (x_ {ip}) \ quad, i = 1, \ ldots, n}

.

Assuming that the estimation of the regression parameters and the error variance is performed using the least squares method , the result is the following least squares minimization criterion:

{\ displaystyle Q = \ sum _ {i = 1} ^ {n} \ left (Y_ {i} - \ beta _ {0} - \ beta _ {1} \ phi _ {1} (x_ {i1}) - \ ldots - \ beta _ {p} \ phi _ {p} (x_ {ip}) \ right) ^ {2} \ rightarrow \ mathrm {Min!}}

.

From this one can easily see that the "linear" aspect of the model means:

The function to be minimized is a quadratic function of the regression coefficients .

{\ displaystyle \ beta _ {j}}

The derivatives of the function are linear functions of that make it easy to find the parameter estimates . ${\ displaystyle \ beta _ {j}}$

The parameter estimates are linear functions of the random variables .

{\ displaystyle b_ {0}, \ ldots, b_ {n}}

{\ displaystyle Y_ {i}}

literature

Ludwig Fahrmeir, Thomas Kneib, Stefan Lang: Regression: Models, Methods and Applications. 2nd Edition. Springer Verlag, 2009, ISBN 978-3-642-01836-7 .