ROC curve

from Wikipedia, the free encyclopedia

The ROC curve ( ROC : English for receiver operating characteristic or German operating characteristic of an observer ), and limit the optimization curve or Isosensitivitätskurve known is a method of evaluation and optimization of analytical strategies. The ROC curve visually represents the dependence of the efficiency on the error rate for various parameter values. It is an application of signal discovery theory .

The ROC curve can be used to find the best possible value of a parameter, for example in the case of a dichotomous (semi-) quantitative characteristic or two-class classification problem.

Calculation of the ROC curve

Interpretation of a ROC curve: Depending on the position of the classifier (vertical line), the proportions of the samples classified as TP- true positive , TN- true negative , FP- false positive , FN- false negative , change from two basic distributions (red for target class , blue comparison class)
ROC curve (real example)

For each possible parameter value (e.g. transmission speed, frequency, ...) the resulting relative frequency distributions are determined in the form of sensitivity (correct positive rate) and false positive rate . Enter the sensitivity (correct positive rate) as the ordinate (" axis") and the false positive rate as the abscissa (" axis") in a diagram . The parameter value itself does not appear, but can be used to label the points. The result is typically a curved, ascending curve.

Interpretation of the ROC curve

An ROC curve close to the diagonal indicates a random process: Values ​​close to the diagonal mean an equal hit rate and a false positive rate, which corresponds to the expected frequency of hits from a random process. The ideal ROC curve initially rises vertically (the hit rate is close to 100%, while the error rate initially remains close to 0%), only then does the false positive rate increase. An ROC curve that remains well below the diagonal indicates that the values ​​have been misinterpreted.

Use as an optimization method

The theoretical optimum (in the sense of a compromise between hit and error rate) of the tested value is then determined visually from the contact point of a 45 ° rising tangent with the ROC curve, provided the axes have been scaled uniformly. Otherwise the tangent rise must be the same as that of the diagonal.

If you draw the test values ​​(for example, depending on the false-positive rate) in the same diagram, the limit value is found as the perpendicular of the contact point of the tangent on the test value curve. Alternatively, the points of the curve can be labeled with the test value. Mathematically, one looks for the test value with the highest Youden index . This is calculated from (calculated with relative values).

An alternative method, which is mainly used in information retrieval , is the consideration of recall and precision .

Use as a quality measure

A ROC curve can also be used as a quality measure. This is often the case in the area of information retrieval . In order to be able to evaluate independently of the test value, the ROC curve is calculated for all or a sample of test values.

The area under the limit value optimization curve or area under the ROC curve ( AUROC for short ) is calculated for the ROC curve . This value can range from 0 to 1, but 0.5 is the worst value. As previously described, an ROC curve near the diagonal is the expected result of a random process that has an area of ​​0.5. The curve previously described as optimal has an area between 0.5 and 1. The curve with the area less than 0.5 can ultimately be just as good in information theory if the result is interpreted in reverse ("positive" and "negative") reversed).

The decisive advantage of using the area under the limit value optimization curve compared to the pure misclassification rate , for example , is that the parameter value is omitted here, while the latter can only ever be calculated for a single specific parameter value. A high AUROC value clearly means “if the parameter is selected appropriately, the result is good”.

example

In information retrieval , the quality of a search result can be evaluated here, for example. "Positive" is a suitable search result, "Negative" is an unsuitable one. The test value is the number of search results requested. If the database contains 10 relevant and 90 irrelevant documents and a method has found 7 relevant documents in the first 12 results, the ROC curve goes through the point . This is calculated for all possible numbers of results (0-100).

The problem as an optimization problem would be: "What is the optimal number of results that I should consider?"

The problem as a quality measure would be: "Regardless of how many results I want to get, how good is the search function?"

In this example, of course, both questions only make limited sense.

Intuition in machine learning

In machine learning , ROC curves are used to evaluate the classifier performance. The misclassification rate is determined for a growing number of instances, starting with the instances for which the classifier is safest (for example because they are the greatest distance from the separation function of a support vector machine ).

As an example, you can imagine an examiner who first lets the examinee answer the questions about which he feels most confident. In the course of the test, the tester can create an ROC curve. Good test subjects then only give incorrect answers at the end of the test, which can be easily read from the ROC curve.

See also

literature