Identifiability

This article covers a technical definition of the term identifiability. For a less technical description, see identification problem

In statistics, and in econometrics in particular, the identifiability of a model is the property of estimation models that inferential statistics can be applied to them.

A model is identifiable if it is theoretically possible to determine the true values on which the model is based by making (drawing) an infinite number of observations. Mathematically, this means that different values ​​of the parameters of the model produce different probability functions of the observable variables.

In practice, where there are a finite number of observations, the identifiability of a model is limited by the number of parameters to be estimated , the number of observations and the number of associated degrees of freedom .

History of the term

The term identifiability was coined by the econometrician Tjalling Koopmans around 1945 with reference to the economic identity of a relationship within a relationship system. The term immediately appeared in the econometrics literature, although Koopman's own account of the topic - his "identification problems in economic model making" - did not appear until 1949. Around 1950 the term was picked up by statisticians and used in a more general sense, see e.g. B. Jerzy Neyman's Existence of Consistent Estimates of the Directional Parameter in a Linear Structural Relation Between Two Variables .

definition

Let be a statistical model with a (possibly infinite-dimensional) parameter space . Then it means identifiable if the mapping is injective . So it should apply: ${\displaystyle {\mathcal {P}}=\{P_{\theta }:\theta \in \Theta \}}$ ${\displaystyle \Theta }$ ${\displaystyle {\mathcal {P}}}$ ${\displaystyle \theta \mapsto P_{\theta }}$

${\displaystyle P_{\theta _{1}}=P_{\theta _{2}}\quad \Rightarrow \quad \theta _{1}=\theta _{2}\;\ {\text{für alle}}\quad \ \theta _{1},\theta _{2}\in \Theta }$.

Different values ​​of should therefore correspond to different probability distributions . ${\displaystyle \theta }$

If the distributions are defined via probability density functions, then these are considered to be different if they differ on a set of positive Lebesgue measures . (For example, two functions that differ in only one point are not considered different probability density functions in this sense.)

This identifiability of the model in the sense of the invertibility of is equivalent to the fact that the true parameters of the model can be determined if one can observe the model indefinitely. For if is the consequence of observations, then it follows from the strong law of large numbers ${\displaystyle \theta \mapsto P_{\theta }}$ ${\displaystyle \{X_{t}\}\subseteq S}$

${\displaystyle {\frac {1}{T}}\sum _{t=1}^{T}\mathbf {1} _{\{X_{t}\in A\}}\ {\xrightarrow {\text{a. s.}}}\ P(X_{t}\in A),}$

for each measurable quantity , where the indicator function denotes a quantity. With an infinite number of observations, one can therefore determine the true probability distribution and, because the mapping is invertible, also the true value of the parameter . ${\displaystyle A\subset S}$ ${\displaystyle \mathbf {1} _{\{...\}}}$ ${\displaystyle P_{\theta }}$ ${\displaystyle \theta \mapsto P_{\theta }}$ ${\displaystyle \theta }$

Examples

Normal distributions

Be the family of normal distributions , a location-scale family forms ${\displaystyle {\mathcal {P}}}$

${\displaystyle {\mathcal {P}}={\Big \{}\ f_{\theta }(x)={\tfrac {1}{{\sqrt {2\pi }}\sigma }}e^{-{\frac {1}{2\sigma ^{2}}}(x-\mu )^{2}}\ {\Big |}\ \theta =(\mu ,\sigma ):\mu \in \mathbb {R} ,\,\sigma \!>0\ {\Big \}}}$.

Then

${\displaystyle {\begin{aligned}&f_{\theta _{1}}=f_{\theta _{2}}\\[6pt]\Longleftrightarrow {}&{\frac {1}{{\sqrt {2\pi }}\sigma _{1}}}\exp \left(-{\frac {1}{2\sigma _{1}^{2}}}(x-\mu _{1})^{2}\right)={\frac {1}{{\sqrt {2\pi }}\sigma _{2}}}\exp \left(-{\frac {1}{2\sigma _{2}^{2}}}(x-\mu _{2})^{2}\right)\\[6pt]\Longleftrightarrow {}&{\frac {1}{\sigma _{1}^{2}}}(x-\mu _{1})^{2}+\ln \sigma _{1}={\frac {1}{\sigma _{2}^{2}}}(x-\mu _{2})^{2}+\ln \sigma _{2}\\[6pt]\Longleftrightarrow {}&x^{2}\left({\frac {1}{\sigma _{1}^{2}}}-{\frac {1}{\sigma _{2}^{2}}}\right)-2x\left({\frac {\mu _{1}}{\sigma _{1}^{2}}}-{\frac {\mu _{2}}{\sigma _{2}^{2}}}\right)+\left({\frac {\mu _{1}^{2}}{\sigma _{1}^{2}}}-{\frac {\mu _{2}^{2}}{\sigma _{2}^{2}}}+\ln \sigma _{1}-\ln \sigma _{2}\right)=0\end{aligned}}}$.

This expression is zero almost everywhere if and only if all of its parameters are zero, which is only possible for and . Because of the scale parameter is positive, the model can be identified: . ${\displaystyle \vert \sigma _{1}\vert =\vert \sigma _{2}\vert }$ ${\displaystyle \mu _{1}=\mu _{2}}$ ${\displaystyle \sigma }$ ${\displaystyle f_{\theta _{1}}=f_{\theta _{2}}\Leftrightarrow \theta _{1}=\theta _{2}}$

Multiple linear regression model

Let that be the classic model of linear multiple regression , with the vector of the unknown regression parameters , the experiment plan matrix , the vector of the dependent variables and the vector of the disturbance variables . Then the parameter can be identified exactly when the matrix is invertible . ${\displaystyle {\mathcal {P}}}$ ${\displaystyle \mathbf {y} =\mathbf {X} {\boldsymbol {\beta }}+{\boldsymbol {\varepsilon }}}$ ${\displaystyle {\boldsymbol {\beta }}}$ ${\displaystyle p\times 1}$ ${\displaystyle n\times p}$ ${\displaystyle \mathbf {X} }$ ${\displaystyle n\times 1}$ ${\displaystyle \mathbf {y} }$ ${\displaystyle n\times 1}$ ${\displaystyle {\boldsymbol {\varepsilon }}}$ ${\displaystyle {\hat {\boldsymbol {\beta }}}}$ ${\displaystyle (\mathbf {X} ^{\top }\mathbf {X} )^{-1}}$

Classic error-in-variables model

Be the classic error-in-the-variables model ${\displaystyle {\mathcal {P}}}$

${\displaystyle {\begin{cases}y=\beta x^{*}+\varepsilon ,\\x=x^{*}+\eta ,\end{cases}},}$

where jointly normally distributed independent random variables with expected value zero and unknown variance are and only the variables are observed. ${\displaystyle (\varepsilon ,\eta ,x^{*})}$ ${\displaystyle (x,y)}$

This model cannot be identified. However, the product (where the variance is the latent regressor ) is identifiable. ${\displaystyle \beta \sigma _{*}^{2}}$ ${\displaystyle \sigma _{*}^{2}}$ ${\displaystyle x^{*}}$

In this example, the exact value of cannot be identified, but it can be guaranteed that it must lie in the interval , where and are the coefficients obtained from on and on by means of an ordinary least-squares estimate . ${\displaystyle {\hat {\beta }}}$ ${\displaystyle ({\hat {\beta }}_{yx},{\frac {1}{{\hat {\beta }}_{xy}}})}$ ${\displaystyle {\hat {\beta }}_{yx}}$ ${\displaystyle {\hat {\beta }}_{yx}}$ ${\displaystyle y}$ ${\displaystyle x}$ ${\displaystyle x}$ ${\displaystyle y}$

literature

• Hans-Friedrich Eckey, Reinhold Kosfeld, Christian Dreger: Econometrics: Basics, Methods, Examples . Gabler Verlag, 2004, ISBN 978-3-409-33732-8 , pp. 321 ( limited preview in Google Book search).

Individual evidence

1. ^ Earliest Known Uses of Some of the Words of Mathematics: Identifiability