Estimation quality for categorical insolvency prognoses

from Wikipedia, the free encyclopedia

Estimation quality measures for categorical insolvency forecasts measure the quality of categorical insolvency forecasts . Insolvency forecasts are referred to as categorical insolvency forecasts that only know two possible forms for assessing the rated companies: "Company A is likely to fail (within the next year)" vs. "Company B is expected (within the next year) does not fail."

Types of forecast errors

Contingency table

Categorical insolvency forecasts divide the companies to be assessed into two groups, “likely to be insolvent” vs. "Probably not insolvent". However, none of the insolvency forecasting methods used today is even remotely capable of producing such precise and, at the same time, always correct forecasts. Aside from “random hits” in small samples, categorical insolvency forecasts will therefore always contain errors. There are two possible types of errors: Type I errors - actual failures that were predicted as non-failures (also type I errors , α errors, false negative proportion) - and Type II errors - actual non-failures that when failures were forecast (also type 2 errors , β errors, false positive proportion). It is customary to set type I errors in relation to all actual failures and type II errors in relation to all actual non-failures (see the following figure).

In the literature on insolvency forecasting, the terms 100% - error of the 1st type are also referred to as hit rate (true positive proportion) and errors of the 2nd type are also referred to as false alarm rate.

Classification errors depending on the selected threshold value and the score density functions for insolvent and non-insolvent companies

For example, the unweighted mean of both error rates or a weighted mean can be used as quality measures, whereby the proportions of failures and non-failures in all companies of the sample can be used as weighting factors (" Bayesian total error ") or the costs associated with both types of error (Error type I: credit default costs, error type II: lost credit margin, other "cross-selling business").

Furthermore, there is in all processes that create categorial prediction, a conflict between errors I and II type depending on the parameters of the.. Bankruptcy prediction method can be achieved by 0% of all failures and 100% of all non-defaults - or vice versa - are correctly detected, and Fine adjustment between these extreme points is usually also possible (see the figure below). In view of the infinite number of alternative possibilities, the choice of a specific error-I-II combination is therefore arbitrary and can therefore only be suitable to a limited extent for measuring the quality of a method.

A conceivable justification for the restriction to a single error-I-II-combination in the assessment of the estimation quality of a method would be the use of " optimal " ("cost-minimal") error combinations I. and II. Art. Which errors-I-II- However, the combination is "optimal" is subjectively different - at a bank the costs for a type I error are in relation to a type II error probably much higher than for a supplier to the company - depends on subjectively influenceable secondary conditions (for example the specific design of credit conditions such as interest rates , collateral , guarantees , ...) and is dependent on variables that change over time, for example on the average default rate.

Conversion of categorical to ordinal estimates of quality

Modern insolvency forecast procedures are no longer based on categorical, but on ordinal or cardinal insolvency forecasts . In the meantime, established quality measures have been established especially for ordinal insolvency prognoses . Ordinal estimation quality measures assess the classification performance of an insolvency forecasting procedure not on the basis of a single, arbitrarily determined, error I-II combination, but on the basis of all possible error I-II combinations that can be generated with the forecasting method. Under empirically well-proven assumptions, the ordinal quality of a method can be estimated using a single error-I-II combination as follows:

with AR * ... Accuracy Ratio ,

F I ... Type I error,
F II ... Error II. Art

swell

  1. ↑ Bemmann (2005).
  2. ↑ See, for example, the overviews in Bemmann (2005, pp. 73ff.)
  3. ↑ See for example Swets (1973, p. 995), Engelmann, Hayden, Tasche (2003, p. 13) and OeNB (2004c, p. 21).
  4. ↑ An alternative, possibly more intuitive definition of hit rate would be: Proportion of actually insolvent companies out of all companies that were predicted as insolvent by the forecasting process. An alternative, possibly more intuitive definition of the false alarm rate would be: Share of all "false positives" (non-insolvent companies that were forecast to be insolvent) in all "alarms" (forecasts that claim a company failure). For an overview of further categorical indicators see Swets, Dawes, Monahan (2000, pp. 25f.) And Swets, Pickets (1982, pp. 24ff.).
  5. ↑ see Balcaen, Ooghe (2004, p. 12 and the literature cited there)
  6. ↑ The "Bayesian total error" indicates which proportion of the prognoses is incorrect without distinguishing between errors I and II. This parameter is conceivably unsuitable for assessing the quality of insolvency prognoses, as the naive prognosis “no company will ever become insolvent” can be used to make prognoses with an overall error of close to 0% (equal to the average insolvency rate) and therefore very separable Insolvency prognoses beats, even if they show only a low error of type II, see OeNB (2004a, p. 117ff.).
  7. ↑ see for example Nanda, Pendharkar (2001, p. 155ff.)
  8. ↑ see for example OeNB (2004b, pp. 33, 80)
  9. ↑ Figure based on Deutsche Bundesbank (2003a, p. 73), Engelmann, Hayden, Tasche (2003, p. 5) and OeNB (2004a, p. 107).
  10. ↑ see Ohlson (1980, p. 124ff.)
  11. ↑ see Balcaen, Ooghe (2004, p. 15)
  12. ↑ see Bemmann (2005, p. 27)

literature
  • Nanda, S., Pendharkar, P. (2001): "Linear Models for Minimizing Misclassification Costs in Bankruptcy Prediction", in International Journal of Intelligent Systems in Accounting, Finance and Management, Vol. 10, pp. 155-168
  • Österreichische Nationalbank (Hrsg.) (2004b): "Lending Process and Credit Risk Management " (PDF; 3.5 MB), series of guidelines on credit risk, Vienna
  • Ohlson, JA (1980): "Financial Ratios and the Probabilistic Prediction of Bankruptcy", in Journal of Accounting Research, Vol. 18 (1), pp. 109-131
  • Swets, JA (1973): "The Relative Operating Characteristic in Psychology", in Science, Vol. 182, pp. 990-1,000
  • Swets, JA, Picket, RM (1982): Evaluation of Diagnostic Systems, Methods from Signal Detection Theory, Academic Press, series in cognition and perception, New York et al.
  • Swets, JA, Dawes, RM, Monahan, J. (2000): "Psychological Science Can Improve Diagnostic Decisions", in Psychological Science in the Public Interest, Vol. 1 (1), pp. 1-26