Orthogonal regression

This item has been on the quality assurance side of the portal mathematics entered. This is done in order to bring the quality of the mathematics articles to an acceptable level . Please help fix the shortcomings in this article and please join the discussion !  ( Enter article )

Orthogonal regression. The red lines represent the distances of the measured value pairs from the best-fit straight line.

In the statistics which serves orthogonal regression (more precisely orthogonal linear regression) for calculating a best-fit line for a finite amount metric scaled data pairs by the method of least squares . As in other regression models , the sum of the squared distances from the straight line is minimized. In contrast to other forms of linear regression , orthogonal regression does not use the distances in or direction, but the orthogonal distances. This procedure does not distinguish between an independent and a dependent variable. In this way, unlike linear regression, applications can be handled in which both variables and are subject to measurement errors. ${\displaystyle (x_{i},y_{i})}$ ${\displaystyle (x_{i},y_{i})}$ ${\displaystyle x-}$ ${\displaystyle y-}$ ${\displaystyle x}$ ${\displaystyle y}$

Orthogonal regression is an important special case of Deming regression . It was first used in 1840 in connection with a geodetic problem by Julius Weisbach , introduced into statistics in 1878 by Robert James Adcock and made known in a more general context by WE Deming in 1943 for technical and economic applications.

Calculation path

It will be a straight line

${\displaystyle y=\beta _{0}+\beta _{1}x}$

sought that minimizes the sum of the squared distances from the associated base points on the straight line. Because of this , one calculates these squared distances , the sum of which is to be minimized: ${\displaystyle (x_{i},y_{i})}$ ${\displaystyle (x_{i}^{*},y_{i}^{*})}$ ${\displaystyle y_{i}^{*}=\beta _{0}+\beta _{1}x_{i}^{*}}$ ${\displaystyle (y_{i}-\beta _{0}-\beta _{1}x_{i}^{*})^{2}+(x_{i}-x_{i}^{*})^{2}}$

${\displaystyle SSR=\sum _{i=1}^{n}{\Big (}(y_{i}-\beta _{0}-\beta _{1}x_{i}^{*})^{2}+(x_{i}-x_{i}^{*})^{2}{\Big )}\ \to \ \min _{\beta _{0},\beta _{1},x_{i}^{*}}SSR}$

The following auxiliary values ​​are required for the further calculation:

${\displaystyle {\overline {x}}={\frac {1}{n}}\sum _{i=1}^{n}x_{i}}$     ( arithmetic mean of ) ${\displaystyle x_{i}}$

${\displaystyle {\overline {y}}={\frac {1}{n}}\sum _{i=1}^{n}y_{i}}$     (arithmetic mean of ) ${\displaystyle y_{i}}$

${\displaystyle s_{x}^{2}={\tfrac {1}{n-1}}\sum _{i=1}^{n}(x_{i}-{\overline {x}})^{2}}$     ( Sample variance of ) ${\displaystyle x_{i}}$

${\displaystyle s_{y}^{2}={\tfrac {1}{n-1}}\sum _{i=1}^{n}(y_{i}-{\overline {y}})^{2}}$     (Sample variance of ) ${\displaystyle y_{i}}$

${\displaystyle s_{xy}={\tfrac {1}{n-1}}\sum _{i=1}^{n}(x_{i}-{\overline {x}})(y_{i}-{\overline {y}})}$     ( Sample covariance of ) ${\displaystyle (x_{i},y_{i})}$

This gives the parameters for solving the minimization problem:

${\displaystyle \beta _{1}={\frac {s_{y}^{2}-s_{x}^{2}+{\sqrt {(s_{y}^{2}-s_{x}^{2})^{2}+4s_{xy}^{2}}}}{2s_{xy}}}}$

${\displaystyle \beta _{0}={\overline {y}}-\beta _{1}{\overline {x}}}$

The coordinates of the base points are also calculated ${\displaystyle x}$

${\displaystyle x_{i}^{*}=x_{i}+{\frac {\beta _{1}}{\beta _{1}^{2}+1}}(y_{i}-\beta _{0}-\beta _{1}x_{i})}$

Alternative calculation method

Distance di of a point P (xi; yi) to the straight line y = mx + t

The geometric distance between a measuring point and a best-fit straight line ${\displaystyle d_{i}}$ ${\displaystyle P(x_{i}|y_{i})}$

${\displaystyle f(x)=mx+t}$

can be calculated as follows:

${\displaystyle d_{i}^{2}={\frac {(y_{i}-(mx_{i}+t))^{2}}{1+m^{2}}}}$

We are now looking for the coefficients and with the smallest sum of the error squares. ${\displaystyle m}$ ${\displaystyle t}$

${\displaystyle \min _{m,t}\sum _{i=1}^{N}d_{i}^{2}}$

Calculation of the partial derivative according to ${\displaystyle t}$

the equation

${\displaystyle {\frac {\partial }{\partial t}}\sum _{i=1}^{N}{\frac {(y_{i}-(mx_{i}+t))^{2}}{1+m^{2}}}=0}$

results as a solution

${\displaystyle t={\overline {y}}-m{\overline {x}}}$

In this case, as the mean value of the measurement points designated coordinates. Analogous to this is the mean value of the coordinates of the measuring points. This solution also means that the point always lies on the best-fit straight line. ${\displaystyle {\overline {x}}}$ ${\displaystyle x}$ ${\displaystyle {\overline {y}}}$ ${\displaystyle y}$ ${\displaystyle P({\overline {x}}|{\overline {y}})}$

Calculation of the partial derivative according to ${\displaystyle m}$

the equation

${\displaystyle {\frac {\partial }{\partial \,m}}\sum _{i=1}^{N}{\frac {(y_{i}-(mx_{i}+t))^{2}}{1+m^{2}}}=0}$

gives the following quadratic equation:

${\displaystyle m^{2}S_{xy}+m(S_{xx}-S_{yy})-S_{xy}=0}$

Are there

${\displaystyle S_{xx}=\sum _{i=1}^{N}(x_{i}-{\overline {x}})^{2}\;}$ and ${\displaystyle \;S_{yy}=\sum _{i=1}^{N}(y_{i}-{\overline {y}})^{2}}$

the sums of squares of the measured values ​​of and and ${\displaystyle X}$ ${\displaystyle Y}$

${\displaystyle S_{xy}=\sum _{i=1}^{N}(x_{i}-{\overline {x}})(y_{i}-{\overline {y}})}$

the product sum between and . ${\displaystyle X}$ ${\displaystyle Y}$

Due to the gradient behavior of this parabola, there is one solution for the minimum:

${\displaystyle m={\frac {S_{yy}-S_{xx}+{\sqrt {(S_{xx}-S_{yy})^{2}+4(S_{xy})^{2}}}}{2S_{xy}}}}$

The equation of the geometric best-fit line is thus:

${\displaystyle f(x)=m(x-{\overline {x}})+{\overline {y}}}$

example

f (x) = 0.8 (x - 3.3) + 4.1

	${\displaystyle x_{i}}$	${\displaystyle y_{i}}$	${\displaystyle x_{i}-{\overline {x}}}$	${\displaystyle y_{i}-{\overline {y}}}$	${\displaystyle (x_{i}-{\overline {x}})^{2}}$	${\displaystyle (x_{i}-{\overline {x}})(y_{i}-{\overline {y}})}$	${\displaystyle (y_{i}-{\overline {y}})^{2}}$
P1	1.0	2.0	−2.3	−2.1	5.29	4.83	4.41
P2	2.0	3.5	−1.3	−0.6	1.69	0.78	0.36
P3	4.0	5.0	0.7	0.9	0.49	0.63	0.81
P4	4.5	4.5	1.2	0.4	1.44	0.48	0.16
P5	5.0	5.5	1.7	1.4	2.89	2.38	1.96
total	${\displaystyle 16{,}5}$	${\displaystyle 20{,}5}$	${\displaystyle 0{,}0}$	${\displaystyle 0{,}0}$	${\displaystyle S_{xx}=11{,}8}$	${\displaystyle S_{xy}=9{,}1}$	${\displaystyle S_{yy}=7{,}7}$
Average	${\displaystyle {\overline {x}}=3{,}3}$	${\displaystyle {\overline {y}}=4{,}1}$

${\displaystyle m={\frac {-4{,}1+{\sqrt {4{,}1^{2}+4\cdot 9{,}1^{2}}}}{2\cdot 9{,}1}}}$

It results and the geometric best fit line is therefore as follows: ${\displaystyle m=0{,}8}$

${\displaystyle f(x)=0{,}8(x-3{,}3)+4{,}1}$

Individual evidence

1. ↑ J. Weisbach: Determination of the main strike and main fall of deposits . In: Archives for Mineralogy, Geognosy, Mining and Metallurgy . 14, 1840, pp. 159-174.
2. ↑ D. Stoyan, T. Morel: Julius Weisbach's pioneering contribution to orthogonal linear regression . In: Historia Mathematica . 45, 2018, pp. 75-84.
3. ^ RJ Adcock: A problem in least squares . In: Annals of Mathematics (Ed.): The Analyst . 5, No. 2, 1878, pp. 53-54. JSTOR 2635758 . doi : 10.2307 / 2635758 .
4. ^ WE Deming: Statistical adjustment of data . Wiley, NY (Dover Publications edition, 1985), 1943, ISBN 0-486-64685-8 .
5. ^ P. Glaister: Least squares revisited . The Mathematical Gazette . Vol. 85 (2001), pp. 104-107.
6. ↑ Casella, G., Berger, RL: Statistical Inference . 2nd Edition. Cengage Learning, Boston 2008, ISBN 978-0495391876
7. ↑ J. Hedderich, Lothar Sachs : Applied statistics. Toolkit with R . 15th edition. Springer Berlin, Heidelberg 2015, ISBN 978-3662456903