Reliability-validity dilemma

from Wikipedia, the free encyclopedia

Reliability-validity dilemma or dilution paradox is a term used primarily in psychological test theory that describes an inverted U-shaped relationship between the reliability and validity of a test. Although reliability is a prerequisite for validity, the dilemma suggests that perfect reliability comes at the expense of validity. The dilemma can not be resolved on the basis of classical test theory .

Derivation

If you want to record the change in a feature between two measurement times, the reliability of the difference values ​​of the two measurements is lower, the higher the correlation between the two measurements, while conversely, the lower the correlation between the two measurements, the lower the validity of the two measurements. This in turn leads to an increase in the reliability of the measured value differences (reliability-validity dilemma in process diagnostics / change measurement).

The reason for the dilemma, however, is not a mathematical one: a clinical thermometer is very reliable and very valid for measuring body temperature. The reason for the reliability-validity dilemma is rather a genuinely psychological one: very homogeneous tests usually deliver very reliable results. Cronbach's alpha is the lower limit of reliability. However, very homogeneous tests can only depict very narrow constructs . However, since many psychological constructs are very complex, very homogeneous (and thus also very reliable) tests are not very suitable for mapping complex constructs or predicting complex criteria.

solution

By using a test battery of many homogeneous sub-tests (= high reliability), each individual test depicts a very reliable, but very narrow facet of the construct. The totality of the subtests represent the complex construct with the help of which one can make good prognoses (= high validity).

See also

literature