Sample covariance

This article covers the covariance of a data series or sample; for the covariance of two random variables see covariance (stochastics) .

This article or the following section is not adequately provided with supporting documents ( e.g. individual evidence ). Information without sufficient evidence could be removed soon. Please help Wikipedia by researching the information and including good evidence.

The sample covariance or empirical covariance (often simply covariance (from the Latin con = "with-" and variance from variare = "(ver) change, be different")) is a non- standardized measure of the (linear) relationship between two in statistics statistical variables . The adjusted sample covariance is an unbiased estimate of the population covariance using a sample.

If the covariance is positive, then small values ​​of one variable are predominantly associated with small values ​​of the other variable and also for large values. For a negative covariance it is exactly the opposite.

definition

If a data series ( sample ) of two statistical variables is and , then the sample covariance is defined as the " average deviation product " ${\displaystyle (x_{1},y_{1}),...,(x_{n},y_{n})}$ ${\displaystyle X}$ ${\displaystyle Y}$

${\displaystyle s_{xy}:={\tfrac {1}{n}}SP_{xy}={\frac {1}{n}}\sum _{i=1}^{n}(x_{i}-{\bar {x}})(y_{i}-{\bar {y}})}$

with the arithmetic means and the data series and . ${\displaystyle \textstyle {\bar {x}}={\frac {1}{n}}\sum _{i=1}^{n}x_{i}}$ ${\displaystyle \textstyle {\bar {y}}={\frac {1}{n}}\sum _{i=1}^{n}y_{i}}$ ${\displaystyle (x_{1},...,x_{n})}$ ${\displaystyle (y_{1},...,y_{n})}$

The sample covariance measures the common dispersion ("co-dispersion") of the observation data of a sample. The mean deviation of the observation data from the mean values ​​is calculated. ${\displaystyle ({\bar {x}},{\bar {y}})}$

The corrected sample covariance is also often used:

${\displaystyle {\hat {\sigma }}_{xy}:={\frac {1}{n-1}}\sum _{i=1}^{n}(x_{i}-{\bar {x}})(y_{i}-{\bar {y}})}$

Construction of the covariance

Construction of the covariance.

The blue data point at the top right in the graphic has a positive contribution to the covariance:

${\displaystyle \underbrace {(x_{i}-{\bar {x}})} _{>0}\times \underbrace {(y_{i}-{\bar {y}})} _{>0}>0}$.

This applies to all data points in quadrant I, with and . These considerations can be continued analogously for the data points in the other quadrants: ${\displaystyle x_{i}>{\bar {x}}}$ ${\displaystyle y_{i}>{\bar {y}}}$

• Data points in quadrant I: positive contribution to covariance,
• Data points in quadrant II: negative contribution to covariance,
• Data points in quadrant III: positive contribution to covariance and
• Data points in quadrant IV: negative contribution to covariance.

If there is a "positive" relationship between the data points, then most of the data points (as in the example on the right) will lie in quadrants I and III and make many positive contributions to the covariance. Although the few data points in quadrants II and IV make negative contributions, the positive contributions will predominate; H. the covariance is positive. If there is a "negative" relationship, it follows with the same reasoning that the covariance is negative.

Corrected sample covariance

To obtain an estimate of the unknown covariance of the population from a sample , the corrected sample covariance is used: ${\displaystyle \sigma _{xy}}$

${\displaystyle {\hat {\sigma }}_{xy}={\frac {1}{n-1}}\sum _{i=1}^{n}{(x_{i}-{\bar {x}})(y_{i}-{\bar {y}})}}$

For a simple random sample , the sample variables and have covariance . Assuming a two-dimensional normal distribution of the sample variables and using the maximum likelihood method , the estimation function results ${\displaystyle X_{i}}$ ${\displaystyle Y_{i}}$ ${\displaystyle \operatorname {Cov} (X_{i},Y_{i})=\sigma _{xy}}$ ${\displaystyle (X_{i},Y_{i})}$

${\displaystyle S_{XY}={\frac {1}{n}}\sum _{i=1}^{n}(X_{i}-{\bar {X}})(Y_{i}-{\bar {Y}})}$.

However, it turns out that the expectation is, i. H. the estimator is not unbiased (i.e. distorted ) for . ${\displaystyle \operatorname {E} (S_{XY})={\tfrac {n-1}{n}}\sigma _{xy}}$ ${\displaystyle S_{XY}}$ ${\displaystyle \sigma _{xy}}$

However, the adjusted sample covariance is unbiased. In the context of inductive statistics , the corrected sample covariance is therefore always used.

Sample covariance vs. Corrected sample covariance

In the context of descriptive statistics , the question arises whether it is better to use the factor or . In general, it depends on the goal of the analysis (or the properties of the sample). ${\displaystyle {\tfrac {1}{n-1}}}$ ${\displaystyle {\tfrac {1}{n}}}$

• If the goal is to estimate the covariance of a population, then the factor must be used because of the unambiguous property . But it should be possible to draw conclusions about the population, e.g. B. the sample could be a simple random sample. ${\displaystyle {\hat {\sigma }}_{xy}}$ ${\displaystyle {\tfrac {1}{n-1}}}$
• If the goal is to describe the data only descriptively, then you can use or . This is e.g. This is the case, for example, when inferences about the population are not wanted or possible. Then the user has to decide which property is more important to him: the possible inference about the population (with ) or the interpretation as mean deviation from (with ). ${\displaystyle {\hat {\sigma }}_{xy}}$ ${\displaystyle s_{xy}}$ ${\displaystyle {\hat {\sigma }}_{xy}}$ ${\displaystyle ({\bar {x}},{\bar {y}})}$ ${\displaystyle s_{xy}}$

In the case of large sample sizes, the difference between and is small anyway, so that the above consideration only has to be made for small sample sizes. ${\displaystyle {\hat {\sigma }}_{xy}}$ ${\displaystyle s_{xy}}$

properties

The following properties apply to both sample covariance and corrected sample covariance.

Interpretation of the covariance

• The covariance is positive if and tend to have a linear relationship in the same direction, i.e. H. high values ​​of are associated with high values ​​of and low values with low values . ${\displaystyle X}$ ${\displaystyle Y}$ ${\displaystyle X}$ ${\displaystyle Y}$
• The covariance, on the other hand, is negative if and have an opposing linear relationship, i.e. H. high values ​​of one variable are associated with low values ​​of the other variable. ${\displaystyle X}$ ${\displaystyle Y}$
• If the result is 0, there is no linear relationship between the two variables and (non-linear relationships are possible). ${\displaystyle X}$ ${\displaystyle Y}$

The covariance indicates the direction of a relationship between two variables, but no statement can be made about the strength of the relationship due to the linearity of the covariance. In order to make a relationship comparable , the covariance must be normalized. The most common normalization using the standard deviation leads to the correlation coefficient .

Relationship to variance

The covariance is an extension of the variance because it holds

• ${\displaystyle s'^{2}=s_{xx}}$ or.
• ${\displaystyle s^{2}={\hat {\sigma }}_{xx}}$.

Where and is the empirical variances with a suitable prefactor. That is, the variance is the covariance of a variable with itself. ${\displaystyle s'^{2}}$ ${\displaystyle s^{2}}$

Displacement set

The shift theorem provides an alternative representation of the covariance

• ${\displaystyle s_{xy}={\frac {1}{n}}\left(\sum _{i=1}^{n}{(x_{i}y_{i})-n{\bar {x}}{\bar {y}}}\right)}$
• ${\displaystyle {\hat {\sigma }}_{xy}={\frac {1}{n-1}}\left(\sum _{i=1}^{n}{(x_{i}y_{i})-n{\bar {x}}{\bar {y}}}\right)}$

. In many cases, these formulas make it easier to calculate the covariance. In the case of numerical calculations, however, attention must be paid to undesired digit deletion when subtracting large numbers.

Symmetry and linearity

The covariance is linear and symmetric; H. the following applies:

symmetry

Swapping the roles of and results in the same value for the covariance: ${\displaystyle x_{i}}$ ${\displaystyle y_{i}}$

• ${\displaystyle s_{xy}=s_{yx}}$ or.
• ${\displaystyle {\hat {\sigma }}_{xy}={\hat {\sigma }}_{yx}}$

Linearity

If one of the variables is subjected to a linear transformation, e.g. B. , then applies ${\displaystyle u_{i}=a+b\cdot x_{i}}$

• ${\displaystyle s_{uy}=bs_{xy}}$ or.
• ${\displaystyle {\hat {\sigma }}_{uy}=b{\hat {\sigma }}_{yx}}$

Because of the symmetry, the covariance is also linear in the second argument.

The linearity of the covariance means that the covariance depends on the unit of measurement of the variable. For example, you get ten times the covariance if you look at the variable instead . Since this property makes the absolute values ​​of the covariance difficult to interpret, one often considers the scale-independent correlation coefficient instead . ${\displaystyle X}$ ${\displaystyle 10X}$

Examples

1.) The following graphic shows the scatter diagram for 21 different data sets together with the covariance and correlation of the data set. The first row shows seven data sets with varying degrees of linear correlation, with the correlation going from +1 to 0 to −1. Since the covariance is a non-standardized measure, it goes from +2 to zero to −2. That is, if there is no linear relationship, then the covariance is just as zero as the correlation. The sign of the covariance indicates the direction of the relationship; however, it does not show the strength of the context. ${\displaystyle s_{xy}}$ ${\displaystyle r_{xy}}$ ${\displaystyle r_{xy}}$

It becomes even clearer in the second line, where all seven data sets have a perfect linear relationship. But the covariance decreases to zero and then becomes negative. The correlation for these data records is either +1 or −1 (or undefined). Finally, the third line shows that both the covariance and the correlation are zero, although there is a clear connection between the two variables. This means that the covariance only measures the linear relationship and non-linear relationships are not recognized. ${\displaystyle s_{xy}}$ ${\displaystyle r_{xy}}$

2.) In a school it should be checked whether there is a connection between the number of hours taught by the teachers per day and the number of cups of coffee drunk. Ten data pairs were collected and evaluated (not carried out like this, just for the sake of illustration!):

number	1	2	3	4th	5	6th	7th	8th	9	10
Number of hours ( ) ${\displaystyle x_{i}}$	5	6th	8th	4th	6th	6th	5	7th	5	4th
Number of cups ( ) ${\displaystyle y_{i}}$	2	1	4th	1	2	0	2	3	3	1

The covariance is now calculated as follows:
a.) First, the arithmetic mean of both variables is determined: and

${\displaystyle \textstyle {\bar {x}}={\frac {1}{n}}\sum _{i=1}^{n}x_{i}={\frac {(5+6+8+4+6+6+5+7+5+4)}{10}}=5{,}6}$ ${\displaystyle \textstyle {\bar {y}}={\frac {1}{n}}\sum _{i=1}^{n}y_{i}={\frac {(2+1+4+1+2+0+2+3+3+1)}{10}}=1{,}9}$

b.) The covariance is now calculated using: ${\displaystyle s_{xy}:={\frac {1}{n}}\sum _{i=1}^{n}(x_{i}-{\bar {x}})(y_{i}-{\bar {y}})}$

${\displaystyle ={\tfrac {(5-5{,}6)(2-1{,}9)+(6-5{,}6)(1-1{,}9)+(8-5{,}6)(4-1{,}9)+(4-5{,}6)(1-1{,}9)+(6-5{,}6)(2-1{,}9)+(6-5{,}6)(0-1{,}9)+(5-5{,}6)(2-1{,}9)+(7-5{,}6)(3-1{,}9)+(5-5{,}6)(3-1{,}9)+(4-5{,}6)(1-1{,}9)}{10}}}$

${\displaystyle =0{,}76}$

Since the covariance is greater than zero, a positive relationship between the number of lessons and the number of cups of coffee can be seen for this sample. Whether this can be generalized to the population, here the teaching staff, depends on the quality of the sample.

See also

• Covariance matrix