Scoring rule

In the decision theory is a score function or scoring rule to German a valuation rule , a measure of the performance of an individual, when afflicted decisions with uncertainty. The weather forecast can be used as an example. A rain probability is generated for each day. Statistics from the forecasts can be used to compare the actual frequency of rain with the forecast. If the forecast is often wrong, it is said to be poorly calibrated . If the predictor can be motivated to improve, then a function can be used where that is the prediction and when it rains and when it doesn't. If the predictor now wants to optimize his performance with this function, then the following function is used to maximize; ${\displaystyle u(x,q)}$ ${\displaystyle q}$ ${\displaystyle x=1}$ ${\displaystyle x=0}$

${\displaystyle {\hat {u}}(u|p)=pu(1,q)+(1-p)u(0,q)\,}$

where p is the predictor's personal probability that it will rain.

Proper score functions

A scoring rule is called proper , i.e. clean , if it only depends on the probability, i.e. the predictor is motivated to estimate honestly and coherently. Two of the most commonly used scoring rules are: The Brier score , given by ${\displaystyle u(x,q)}$ ${\displaystyle {\hat {u}}(x,q)}$ ${\displaystyle 0\leq p\leq 1}$

${\displaystyle u(x,q)=1-(x-q)^{2}\,}$

and the logarithmic score function .

${\displaystyle u(x,q)={\begin{cases}\log q&{\text{falls }}x=1\\\log(1-q)&{\text{falls }}x=0\\\end{cases}}}$

Further scoring rules

These are examples of strictly proper scoring rules:

The quadratic score :

${\displaystyle u(x,q)=2x-q^{2}}$

The spherical scoring rule :

${\displaystyle u(x,q)=x/{\sqrt {q^{2}}}}$

Web links

• Video comparing spherical, quadratic and logarithmic scoring rules