Smit's paradox

In classic mathematical statistics, the following applies, to put it casually: the larger the sample , the better the estimate (for more details, see main theorem of mathematical statistics ). In the statistics of random processes , however, it is possible - and is then usually perceived as paradoxical - that an estimate becomes worse as the sample size increases . SJ Wilenkin was the first to notice this in 1959, but there were mistakes in his work, so that JC Smit became the namesake of the paradox in 1961.

The paradox

Let be a weakly stationary random process with an unknown constant expectation value and a (known) covariance function . The process can be watched for Be (discrete) observations and continuous observation of the process over the whole . Than are ${\displaystyle X_{t}}$ ${\displaystyle \mathrm {E} X_{t}=m}$ ${\displaystyle r(t-s)}$ ${\displaystyle t\in T\subset \mathbb {R} }$ ${\displaystyle \quad x_{t_{1}},\cdots ,x_{t_{n}};\quad t_{i}\in T\quad n}$ ${\displaystyle x_{t},t\in T}$ ${\displaystyle T}$

${\displaystyle {\hat {m}}={\frac {1}{n}}\sum _{i=1}^{n}x_{t_{i}};\quad \quad {\tilde {m}}={\frac {1}{|T|}}\int _{T}x_{t}\mathrm {d} t}$

unbiased estimates for . Intuitively it seems to be clear that is better than because it uses more information, namely information from the whole , while only uses point information. But even for simple special cases, the opposite is shown: it is better than if one takes the variance of the estimators as a criterion: ${\displaystyle m}$ ${\displaystyle {\tilde {m}}}$ ${\displaystyle {\hat {m}}}$ ${\displaystyle T}$ ${\displaystyle {\hat {m}}}$ ${\displaystyle {\hat {m}}}$ ${\displaystyle {\tilde {m}}}$

${\displaystyle \operatorname {Var} ({\hat {m}})={\frac {1}{n^{2}}}\sum _{i,j=1}^{n}r(t_{i}-t_{j});\quad \operatorname {Var} ({\tilde {m}})={\frac {1}{|T|^{2}}}\int _{T}\int _{T}r(t-s)\mathrm {d} t\mathrm {d} s}$

example

Be , d. H. discreet observatories. Then it follows , i. H. is better than . If one includes further observations between the previous places, i. H. at , then the variance of from worsens to , i.e. h, a “condensation” of the observations leads to a worse result. ${\displaystyle T=[-1,1];\quad r(t-s)=\mathrm {e} ^{-|t-s|};\quad t_{1}=-1,\quad t_{2}=-0{,}5,\quad t_{3}=0,\quad t_{4}=0{,}5,\quad t_{5}=1}$ ${\displaystyle n=5}$ ${\displaystyle \operatorname {Var} ({\hat {m}})=0{,}529;\quad \operatorname {Var} ({\tilde {m}})=0{,}568)}$ ${\displaystyle {\hat {m}}}$ ${\displaystyle {\tilde {m}}}$ ${\displaystyle t_{6}=-0{,}75,\quad t_{7}=-0{,}25,\quad t_{8}=0{,}25,\quad t_{9}=0{,}75}$ ${\displaystyle {\hat {m}}}$ ${\displaystyle 0{,}529}$ ${\displaystyle 0{,}542}$

Resolution of the paradox

The estimate is for not the best linear unbiased estimate ( English Best Linear Unbiased Estimator , BLUE for short ), so it is compared with a non-optimal estimate. The BLUE for results after a set of Grenander in the form of a Stieltjesintegrales as a solution of the integral equation with . ${\displaystyle {\tilde {m}}}$ ${\displaystyle m}$ ${\displaystyle {\hat {m}}}$ ${\displaystyle m}$ ${\displaystyle {\tilde {m}}^{*}=\int _{T}x_{t}\mathrm {d} G^{*}(t);\quad \int _{T}\mathrm {d} G^{*}(t)=1}$ ${\displaystyle \int _{T}r(t-s)\mathrm {d} G^{*}(t)=c}$ ${\displaystyle c=\operatorname {Var} ({\tilde {m}}^{*})}$

Continued example

See also. With the same settlements as in the example above, the result is

${\displaystyle {\tilde {m}}^{*}={\frac {1}{4}}[x_{-1}+\int _{-1}^{1}x_{t}\mathrm {d} t+x_{+1}];\quad \operatorname {Var} ({\tilde {m}}^{*})=0{,}500}$.

${\displaystyle {\tilde {m}}^{*}}$in contrast to extra weights, places them on the edge of the observation interval ( ). The discrete five-point estimate approximates this marginal weighting better than and is therefore naturally the better estimator. ${\displaystyle {\tilde {m}}}$ ${\displaystyle t=-1,t=+1}$ ${\displaystyle {\hat {m}}}$ ${\displaystyle {\tilde {m}}}$

Practical meaning

The phenomenon described for stochastic processes also applies to random fields . In geostatistics in particular , it is important to know that network densification in geographic information systems does not automatically lead to better estimation results.

Individual evidence

1. ↑ SJ Wilenkin: Whether ocenke srednego v stacionarnych processach. In: Teorija Verojatnost. IV, 1959, pp. 451-453.
2. ↑ JC Smit: Estimation of the mean of a stationary stochastic process by equidistant observations. In: Trabojos de estadistica. 12, 1961, pp. 35-45.
3. ↑ U. Grenander: Stochastic processes and statistical inference. In: Arkiv för Matematik. 1, 1950, pp. 195-277.
4. ^ W. Näther: Effective Observation of Random Fields. (= Teubner texts on mathematics. Volume 72). Teubner Verlag, Leipzig 1985.
5. ↑ W. Näther: Good and bad examples from the design of experiments for stochastic processes and fields. In: Series of publications by the Institute for Mine Surveying and Geodesy at the TU Bergakademie Freiberg. Issue 2, 2004, pp. 8-19.