# Efficiency (statistics)

The efficiency is a key criterion in mathematical statistics and provides the opportunity to point estimates to compare. The efficiency is not used uniformly in the literature, so the definition of the respective author should always be checked. Some distinctions are:

• Efficiency and asymptotic efficiency , i.e. the occurrence of efficiency only in the limit value.
• Definitions only for unbiased estimators or also for those with bias
• Formulated using the Cramér-Rao inequality , i.e. only in regular statistical models . Accordingly, the term Cramer-Rao efficiency is used.
• "absolute" efficiency versus "relative" efficiency. An absolutely efficient estimator is better than all other estimators in a defined class, a relatively efficient estimator is only better than a specified counterpart.

Correspondingly, there are also combinations of the options listed above. The central comparison criterion in the case of true to expectation is the variance of the estimator, in the case of non-true to expectation the mean square error or, in general, risk functions that are obtained from given loss functions .

In addition to consistency , sufficiency and (asymptotic) faithfulness to expectations, efficiency is one of the four common quality criteria of point estimators .

## idea

The efficiency relates to the variance of an estimator. The smaller the variance of an estimator, the closer an estimated value (on average), calculated from a sample, to which true parameters lie. A distinction is made between relative and absolute efficiency.

If one has two unbiased estimators for the same unknown parameter, then the estimator with the smaller variance is called (relatively) efficient or more efficient . The more efficient estimator would be preferred to solve the estimation problem. The Cramér-Rao inequality states that for many estimation problems there is a lower limit for the variance of an unbiased estimator. Once such an estimator has been found, there is no other unbiased estimator that has a smaller variance. If one can show that an estimator has the minimum variance for an estimation problem, then this estimation function is called absolutely efficient .

### example

For independent sample variables with and , the two estimators ${\ displaystyle X_ {i}}$${\ displaystyle \ operatorname {E} (X_ {i}) = \ mu}$${\ displaystyle \ operatorname {Var} (X_ {i}) = \ sigma ^ {2}}$

${\ displaystyle T_ {1} (X_ {1}, X_ {2}, X_ {3}) = {\ tfrac {1} {3}} (X_ {1} + X_ {2} + X_ {3}) }$

and

${\ displaystyle T_ {2} (X_ {1}, X_ {2}, X_ {3}) = {\ tfrac {1} {3}} (X_ {1} +2 \ cdot X_ {3})}$

for the unknown parameter . ${\ displaystyle \ mu}$

Both estimators are expected faithful: . For the variance, however, results ${\ displaystyle \ operatorname {E} (T_ {j} (X_ {1}, X_ {2}, X_ {3})) = \ mu}$

${\ displaystyle \ operatorname {Var} (T_ {1} (X_ {1}, X_ {2}, X_ {3})) = \ operatorname {Var} ({\ tfrac {1} {3}} (X_ { 1} + X_ {2} + X_ {3})) = {\ frac {1} {9}} \ left (\ operatorname {Var} (X_ {1}) + \ operatorname {Var} (X_ {2} ) + \ operatorname {Var} (X_ {3}) \ right) = {\ frac {1} {3}} \ sigma ^ {2}}$

and

${\ displaystyle \ operatorname {Var} (T_ {2} (X_ {1}, X_ {2}, X_ {3})) = \ operatorname {Var} ({\ tfrac {1} {3}} (X_ { 1} +2 \ cdot X_ {3})) = {\ frac {1} {9}} \ left (\ operatorname {Var} (X_ {1}) + 4 \ cdot \ operatorname {Var} (X_ {3 }) \ right) = {\ frac {5} {9}} \ sigma ^ {2}}$.

This applies

${\ displaystyle \ operatorname {Var} (T_ {1} (X_ {1}, X_ {2}, X_ {3})) = {\ frac {1} {3}} \ sigma ^ {2} <{\ frac {5} {9}} \ sigma ^ {2} = \ operatorname {Var} (T_ {2} (X_ {1}, X_ {2}, X_ {3}))}$,

that is, is more efficient than . ${\ displaystyle T_ {1}}$${\ displaystyle T_ {2}}$

## Mathematical definition

### Expectant case

Formally, assume an unbiased estimator for the unknown parameter in a family of probability densities and the Fisher information belonging to the density . Then the efficiency of is defined as follows: ${\ displaystyle T (X)}$${\ displaystyle \ vartheta}$${\ displaystyle I (\ vartheta)}$${\ displaystyle \ prod _ {i = 1} ^ {n} f _ {\ vartheta} (x_ {i})}$${\ displaystyle T (X) \;}$

${\ displaystyle e (T (X)) = {\ frac {1} {I (\ vartheta) \ mathrm {Var} _ {\ vartheta} (T (X))}}}$.

If you want to compare two unbiased estimators and with each other, the estimator that has the higher value and therefore the smaller variance is called more efficient . ${\ displaystyle T_ {1} (X)}$${\ displaystyle T_ {2} (X)}$${\ displaystyle e (T_ {i} (X))}$

A consequence of the Cramér-Rao inequality is that under regularity conditions there is an upper bound by 1 and therefore such estimators are called efficient (or more precisely Cramér-Rao efficient) for which and thus holds. Under the conditions of the stochastic model necessary for the Cramér-Rao inequality, this is the best possible variance of an estimator. ${\ displaystyle e (T (X))}$${\ displaystyle e (T (X)) = 1}$${\ displaystyle \ mathrm {Var} _ {\ vartheta} (T (X)) = I (\ vartheta) ^ {- 1}}$

### Non-expectation case

If the estimator is not expected, its efficiency can be expressed as ${\ displaystyle T}$

${\ displaystyle e (T (X)) = {\ frac {\ left ({\ frac {\ partial} {\ partial \ vartheta}} E _ {\ vartheta} [T (X)] \ right) ^ {2} } {I (\ vartheta) \ mathrm {Var} _ {\ vartheta} (T (X))}}}$

define. Obviously, the above definition is a special case.

## Asymptotic efficiency

As a rule, it is sufficient if estimators are asymptotically efficient, i.e. H. if they converge in distribution to a normally distributed random variable whose variance is the inverse of the Fisher information. Formally, the convergence statement should be

${\ displaystyle {\ sqrt {n}} (T (X) - \ vartheta) \; {\ overset {\ mathcal {D}} {\ longrightarrow}} \; {\ mathcal {N}} (0, I_ { 1} (\ vartheta) ^ {- 1})}$

can be proven, where the Fisher information of the density denotes and holds. Obviously, for asymptotically efficient estimators${\ displaystyle I_ {1} (\ vartheta)}$${\ displaystyle f _ {\ vartheta} (x)}$${\ displaystyle I (\ vartheta) = n \ cdot I_ {1} (\ vartheta)}$${\ displaystyle \ lim _ {n \ rightarrow \ infty} e (T) = 1.}$

Typical examples of asymptotically efficient estimators are those obtained using the maximum likelihood method .