Quantum reaction equilibrium

The quantum reaction equilibrium , also called quantal response equilibrium ( English Quantal Response Equilibrium , short: QRE ), is a term from game theory . It describes a combination of strategies in non-cooperative games : Each player chooses a strategy from which it makes no sense for any player to deviate from his chosen strategy, although they do not know whether it is the best of the strategies, and Make mistakes. The strategies of the players are therefore, at best, each other's best answers . When the players play completely rationally , the quantum reaction equilibrium converges to the Nash equilibrium . The quantum reaction equilibrium is a statistical solution concept in game theory. The definition and proof of existence of the quantum reaction equilibrium go back to the 1995 paper Quantal Response Equilibria for Normal Form Games by the economists Richard McKelvey and Thomas Palfrey . The quantum reaction equilibrium is of central importance in economic areas such as econometrics and the statistical consideration of inconsistent players, for example in elections or the Traveler's Dilemma . The quantum reaction equilibrium is only defined for games with discrete strategies.

Basic idea

Players choose a strategy that they don't know is the best of all strategies and they make mistakes when choosing. You choose from a discrete strategy space and make assumptions about the choice of the opponent's strategy. This assumption or belief is proven true in expectation . In equilibrium, the assumptions about the opponent's choice of strategy are correct and the errors in the selection are minimal, so that each player's payout is maximized and ensures that errors have no effect on a player's choice of strategy. In the case of perfectly rational individuals, statistical predictions coincide with the theoretical choice of equilibrium, since no one has any incentive to deviate from their choice of equilibrium.

The concept of quantum reaction equilibrium differs from other theoretical equilibrium predictions in that it is a statistical prediction of an equilibrium choice from discrete strategies. By modeling an error term , any behavior of players can be predicted, which fact limits the quality of an equilibrium concept. In principle, statements about the quantum reaction equilibrium can be interpreted as a descriptive equilibrium prediction.

Establishing the quantum reaction equilibrium in normal form

Verbal representation

The strict assumption of the perfect rationality of the players, modeled by the deterministic nature of a strategic game environment, is to be extended by the addition of a stochastic expression. As a result, possible player inconsistency is possible. In this game environment, solutions of the equilibrium can only be expressed in terms of probabilities, as in deterministic game environments, since the equilibrium depends on the stochastic part and a probability of the deterministic choice and is therefore always economically relevant when the benefit of a player on the realization of the random variable is determined.

The expected payout of a player from different strategies depends on the beliefs of the players about the choice of strategy of other players. Beliefs determine expected payouts, which in turn generate options and QR functions. In equilibrium, these beliefs match the equilibrium choices.

Players make " infinitesimal " mistakes. This change makes it possible to observe deviations from perfectly rationally expected gaming behavior and to use logistic regression to formally describe which fact can have significantly different results to predict the Nash equilibrium , but with increasing probability or knowledge about the realization of the random variable against the Nash equilibrium converges.

Compared to the Nash equilibrium, the quantum reaction equilibrium adds an uncertainty factor and thus makes it more resistant and generally valid than the deterministic model, since the behavior of "noisy players" can be modeled. In fact, however, the quantum reaction equilibrium is a generalization of the Nash equilibrium ( NGGW ), which converges to it with decreasing or increasing rationality and approaches the best-answer function . ${\displaystyle \mu }$

This provides a useful theoretical framework for observing the comparative static effects of parameter changes. It does not necessarily lead to deviations from the Nash predictions.

• Players do not choose the best answer with probability 1 (as in the Nash equilibrium)
• Players choose answers with higher expected payouts weighted with higher probability - better answers than best answers
• Players have rational expectations and use true expected errors in interpreting other players' strategies.

In this modeling , players have a free choice according to a good (here: strategy) and decide under stochastic influence. You know the selection probability p and estimate this probability “better” with increasing experience. The deterministic part of a player's choice of strategy is characterized by observable attributes and the stochastic part is subject to unobservable influences.

The player's preference is to choose a strategy that generates a higher expected payout than others, although it is not a given that it is the best of all strategies, as payouts only exist in expected values. One assumption in the model is that players correctly estimate their expected payouts. This means that the player receives his estimate of the payoff from strategy a in the expected value, taking into account the stochastic choice of equilibrium of the other players. The added stochastic variable influencing the discrete choice is endogenous in the model. ${\displaystyle i}$

In contrast to the Nash equilibrium, however, the quantum reaction equilibrium provides the possibility of making statistical prognoses instead of deterministic statements. The quality of these statistical statements depends significantly on the precision of the players' beliefs about the expected payouts of the different strategies. Player experience and learning ability play a particular role as these factors influence the ability to accurately estimate expected payouts from particular strategies. This phenomenon can also be explained by the effect of increasing observations in econometrics.

Formal representation

Assumptions

The normal form of a game with the following elements is:

• ;Player:

There are players, with ${\displaystyle i\in \mathbb {N} }$ ${\displaystyle N=\{1,\ldots ,n\}}$

• ; In the strategy room: ${\displaystyle s_{i}\in S}$

there is a strategy for every player

${\displaystyle S_{ij}={\begin{pmatrix}s_{11}&\ldots &s_{1i}\\\ldots &&\ldots \\s_{i1}&\ldots &s_{ij}\end{pmatrix}}}$

consisting of pure strategies. ${\displaystyle J_{i}}$

• ; Withdrawal function

For every player there is a payout function where ${\displaystyle i\in \mathbb {N} }$ ${\displaystyle u_{i}\colon S\to \mathbb {R} }$

${\displaystyle \prod _{i\in \mathbb {N} }S_{i}}$

• ;Probability

${\displaystyle p_{i}={\mathcal {P}}(X=x_{i})\quad }$and where ${\displaystyle \quad p_{i}\colon S_{i}\to \mathbb {R} }$

${\displaystyle \sum _{s_{ij}\in S_{i}}p_{i}(s_{ij})=1\quad }$and for everyone ${\displaystyle \quad p_{i}(s_{ij})\geq 0}$ ${\displaystyle s_{ij}\in S_{i}}$

For completeness, it should be mentioned that all p are in a space of probabilities in which: ${\displaystyle \Delta }$

All others , with and , so ${\displaystyle {\overline {p}}_{i}}$ ${\displaystyle p_{i}={\mathcal {P}}(X=x_{i})}$ ${\displaystyle {\overline {p}}_{i}=1-{\mathcal {P}}(X=x_{i})}$ ${\displaystyle p_{i}+{\overline {p}}_{i}=1}$

so

${\displaystyle \Delta _{i}=\left\{p_{i}=(p_{i1},\ldots ,p_{iJ}):\sum _{j}p_{ij}=1,p_{ij}\geq 0\right\}}$

• ; Probability player chooses strategy ${\displaystyle i}$ ${\displaystyle s_{ij}}$

${\displaystyle p_{ij}=p_{i}(s_{ij})}$

Therefore the notation represents the strategy, where i chooses the strategy and all other players adapt its expression of p. ${\displaystyle (s_{ij},p_{-i})}$ ${\displaystyle s_{ij}}$

• ; Utility function

${\displaystyle u_{is}=V_{is}+\varepsilon _{is}}$

Where represents the deterministic part and the stochastic part of the model. ${\displaystyle V_{is}}$ ${\displaystyle \varepsilon _{is}}$

Furthermore, it can be described as white noise and provided with the following assumptions: ${\displaystyle \varepsilon _{is}}$

${\displaystyle \mathbb {E} [\varepsilon _{is}]=0}$ ${\displaystyle \forall {is}}$

Both the distribution function and the density function are unknown. The expected value exists and is zero.

In the logit model, however, the stochastic part of the utility function is distributed to extremes , which assumption leads to useful effects. The extreme value distributed error term can be defined as the player 's error vector. ${\displaystyle i}$

The utility function of the players

Deviations from optimal decisions are negatively correlated with associated costs. In other words, gamblers are very reluctant to make high-cost mistakes. Formally, the utility function forms a vector with a deterministic part and a stochastic part with the above-mentioned assumptions. Expected Payouts ${\displaystyle V_{is}}$ ${\displaystyle \varepsilon _{is}}$

${\displaystyle u_{i}(p)=p(s_{1})\cdot u_{i}(s_{1})+p(s_{2})\cdot u_{i}(s_{2})+\ldots =\mathbb {E_{p}} [u_{i}]}$

or more precisely ${\displaystyle u_{i}(p)=\sum p(s)u_{i}(s)}$

in turn are determined by beliefs about the actions of other players and form the deterministic part of the utility function. Player payouts are weighted with the likelihood that the strategy will be played.

Under the assumptions mentioned above, Nash equilibria in pure strategies only exist in expected values ​​and with an error term. The payouts can be through the vector

${\displaystyle {\overline {u}}(p)={\begin{pmatrix}u_{1}(p)\\u_{2}(p)\\\vdots \\u_{j}(p)\end{pmatrix}}}$

Where,

${\displaystyle {\overline {u}}_{ij}(p)=u_{i}(s_{ij},p_{-i})}$

be made more understandable.

The player chooses strategy j when and becomes maximum. In addition, there is the error term , so that for every u there is an {ij} -and set R for every player : ${\displaystyle i}$ ${\displaystyle {\overline {u}}_{ij}\geq {\hat {u}}_{ik}\forall k\in J}$ ${\displaystyle {\overline {u}}_{ij}}$ ${\displaystyle \varepsilon }$ ${\displaystyle i}$

${\displaystyle R_{ij}({\overline {u}}_{i})=\{\varepsilon _{i}\in \mathbb {R} \mid {\overline {u}}_{ij}+\varepsilon _{ij}\geq {\overline {u}}_{ik}+\varepsilon _{ik}\ \forall k=1,\ldots ,J_{i}\}}$

A set , given p, gives the region of errors that leads the player to strategy j: ${\displaystyle R_{ij}({\overline {u}}_{i}(p))}$ ${\displaystyle i}$

${\displaystyle \sigma _{ij}({\overline {u}}_{i})=\int _{R_{ij}({\overline {u}}_{i})}f(\varepsilon )\ \mathrm {d} \varepsilon }$

which is the probability that the player will choose strategy j given and corresponds to the quantum reaction equilibrium. ${\displaystyle i}$ ${\displaystyle {\overline {u}}}$

A probability-weighted payout of all strategies arises, taking into account the characteristics of the residual. There is a random best-answer function for all games in normal form and thus also a quantum reaction equilibrium. The equilibrium choices form the quantum reaction equilibrium. In equilibrium, the players' beliefs are correct. A modeling is possible through the logit equilibrium, since unobserved disturbance terms result in deviations in the benefit of the players and should be kept as small as possible as the aim of the modeling.

Logit quantum reaction equilibrium

The most common indication of a quantum reaction equilibrium ( QRE ) is the logit quantum reaction equilibrium ( English Logit Quantal Response Equilibrium , short: LQRE ):

Gumbel distribution function

The core idea of ​​the logit modeling of the choice of the strategy of players is a discrete decision model . Thus it is possible to make statements regarding the choice of alternative strategies of the players. Player chooses from the strategy space without taking time into account , as this is a one-time game. The player prefers strategy before if: ${\displaystyle i\in \mathbb {N} }$ ${\displaystyle s_{i}\in S}$ ${\displaystyle t\in T}$ ${\displaystyle s}$ ${\displaystyle {\overline {s}}}$

Gumbel density function

${\displaystyle u_{is}\geq u_{i{\overline {s}}}\forall s,{\overline {s}}\in S,s\neq {\overline {s}}}$

Under logit models is meant a form of the binary selection problems in the stochastic part of the utility function of distributed independent and identically error terms that an extreme value distribution follow . According to the Fisher-Tippett theorem (later Fisher-Tippett Gnedenko theorem) these approach an extreme value distribution. ${\displaystyle \varepsilon _{is}}$ ${\displaystyle e\;{\stackrel {\mathrm {i.i.d} }{\sim }}\;{\mathcal {E}}(\lambda ,\gamma )}$

Gumbel realizations (standard)

The distribution function ( over-extreme value distribution ) fulfills these assumptions and is a class of extreme value distribution. The distribution function (graph 1) shows how likely a result is that at most the value on the x-axis is observed. The realizations (graph 3, 4) of this distribution show the observations over time. The Gumbel distribution (0.1) is defined here under Standard and (mu, beta) indicates a differently scaled expression of the realizations. The area under the density function (Figure 2) up to a realization corresponds to the probability at which this value occurs at most.

Gumbel realizations (mu, beta)

Therefore stochastic disturbance terms are to be understood as the maximum of the random numbers . The form of the logit model allows estimates to be made using the maximum likelihood method of the choice of players. The selection probability results from extremely distributed stochastic disturbance terms:

${\displaystyle {\mathcal {P}}_{i}^{MNL}(s|S)={\mathcal {P}}(u_{is}\geq u_{i{\overline {s}}}\forall s\neq {\overline {s}})={\frac {e^{\mu V_{is}(\beta )}}{\sum _{{\overline {s}}\in S}e^{\mu V_{is}(\beta )}}}}$

${\displaystyle \mu \in \mathbb {R} ^{+}}$can in this case be identified as a rationality parameter and indicates how rational the players decide. A player with makes perfectly rational decisions and all predictions converge to the Nash equilibrium. ${\displaystyle \mu \to 0}$

The problem, however, is the assumption of the independently and identically distributed fault terms. The resulting correlation of 0 between the disturbance terms results in a ratio of the selection probabilities.

The ratio for the alternatives is: ${\displaystyle s\neq s^{\dagger }}$

${\displaystyle {\frac {{\mathcal {P}}_{i}^{MNL}(s|S)}{{\mathcal {P}}_{i}^{MNL}(s^{\dagger }|S)}}={\frac {\frac {e^{\mu V_{is}(\beta )}}{\sum _{s\in S}e^{\mu V_{is}(\beta )}}}{\frac {e^{\mu V_{is^{\dagger }}(\beta )}}{\sum _{s^{\dagger }\in S}e^{\mu V_{is^{\dagger }}(\beta )}}}}={\frac {e^{\mu V_{is}(\beta )}}{e^{\mu V_{is^{\dagger }}(\beta )}}}}$

The constancy of this relationship contradicts the independence of irrelevant alternatives ( English Independence of irrelevant alternatives , short: IIA ).

Applications

The quantum reaction equilibrium is used in games with discrete strategies. In Traveler's Dilemma, the quantum reaction equilibrium can explain observed data on player behavior. With a certain choice of the rationality parameter, it is possible to model any behavior of players.

${\displaystyle s_{i,j}}$	${\displaystyle s_{2,1}}$	${\displaystyle s_{2,2}}$
${\displaystyle s_{1,1}}$	(-1.1)	(1, -1)
${\displaystyle s_{1,2}}$	(-1.1)	(1, -1)

Crowd ${\displaystyle N=\{1,2\}}$

Strategy set ${\displaystyle s_{1}\neq s_{2}\in S}$

Strategy set player 1 ${\displaystyle s_{1}=\{T,B\}}$

Strategy amount player 2 ${\displaystyle s_{2}=\{L,R\}}$

Zero-sum game with discrete strategies

Line player's expected payoff function from strategy ( ) is a function of the selection probability of column player's strategy ( ), which can be formed from expected values: ${\displaystyle T}$ ${\displaystyle U_{T}}$ ${\displaystyle R}$ ${\displaystyle p_{R}}$

${\displaystyle U_{T}(p_{R})=p_{R}-(1-p_{R})=2p_{R}-1}$

Graphic representation, application

Analogously, line player's expected payoff function from strategy B is formed from the selection probability of column player of his strategy R:

${\displaystyle U_{B}(p_{R})=1-2p_{R}}$

If column player prefers to play R ( ), line player's best answer is strategy T. ${\displaystyle p_{R}\geq 1/2}$

Column player's expected payouts can be calculated analogously.

The QR function smooths the discontinuous calculated best-answer function and represents monotonous and stochastic choices as a function of payouts. In the graph, the QR function and the best answer function intersect in the Nash equilibrium. With a different rationality parameter, the QR function shifts and different statistical predictions regarding an equilibrium are made. The opponent's QR function is calculated analogously.

Building the quantum reaction equilibrium in extensive form

Verbal representation

In an extensive form of the game, the time factor is included in the model and a kind of step game is created. In the deterministic model, statements about a time-resistant equilibrium can also be made in the infinitely often repeated step game, since a Nash equilibrium must always remain an equilibrium. The stochastic influence and the disturbance term, however, prevent this ability due to the dependence of the equilibrium on the realizations of various random variables. Only expected values ​​can be given which ultimately cannot make reliable predictions. The law of large numbers means that more consistent statements about equilibria can be made with increasing observations. McKelvey and Palfrey define an agent quantum reaction equilibrium ( English Agent Quantal Response Equilibrium , AQRE for short ) for the dynamic game , which can be determined with the help of partial game perfection . In this game, each player determines his expected payout by modeling the future as his own player with knowledge of the probability distribution over the strategies.

Web links

• McKelvey, Palfrey 1995: URL http://www.dklevine.com/archive/refs4510.pdf
• McKelvey, Palfrey 1998: URL http://fisher.osu.edu/~schroeder.9/AMIS900/McKelvey1998.pdf
• Becker, Carter, Naeve 2005: URL https://www.uni-hohenheim.de/RePEc/hoh/papers/252.pdf
• Goeree, Holt, Palfrey: URL http://people.hss.caltech.edu/~jkg/QRE%20Palgrave.pdf
• Economics 209B, Behavioral / Experimental Game Theory: URL http://eml.berkeley.edu/~kariv/209B_QRE.pdf

literature

• McFadden, D., 1973. Conditional logit analysis of qualitative choice behavior
• Fisher, RA, Tippett, LHC, 1928. Limiting forms of the frequency distribution of the largest or smallest member of a sample. In: Mathematical Proceedings of the Cambridge Philosophical Society. Vol. 24. Cambridge Univ. Press
• Train, KE, 2009. Discrete Choice Methods with Simulation

Individual evidence

1. ↑ ^{a b} Becker et al .: Experts Playing the Traveler's Dilemma, Hohenheimer Discussion Contributions, No. 252/2005, p. 13
2. ↑ ^{a b c} Goeree, Holt, Palfrey: Quantal Response Equilibrium, Division of the Humanities and Social Sciences, p. 1.
3. ↑ McKelvey, Palfrey: Quantal Response Equilibria for Normal Form Games , Games and Economic Behavior Vol. 10, page 7, line 2, 1995
4. ↑ McKelvey, Palfrey: Quantal Response Equilibria for Normal Form Games , Games and Economic Behavior Vol. 10, pp. 6-8, 1995
5. ↑ ^{a b} Goeree, Holt, Palfrey: Quantal Response Equilibrium, Division of the Humanities and Social Sciences, p. 2.
6. ↑ Economics 209B, Behavioral / Experimental Game Theory: Lecture 4: Quantal Response Equilibrium (QRE), Spring 2008
7. ^ McKelvey, Palfrey: Quantal Response Equilibria for Normal Form Games , Games and Economic Behavior Vol. 10, page 10, "better actions are more likely to be chosen than worse actions", 1995
8. ^ McKelvey, Palfrey: Quantal Response Equilibria for Normal Form Games , Games and Economic Behavior Vol. 10, p. 7, 1995
9. ↑ McKelvey, Palfrey: Quantal Response Equilibria for Normal Form Games , Games and Economic Behavior Vol. 10, pp. 8 ff., 1995
10. ↑ McKelvey, Palfrey: Quantal Response Equilibria for Normal Form Games , Games and Economic Behavior Vol. 10, p. 10, 1995
11. ^ Train, K. E: Discrete Choice Methods with Simulation, Cambridge University Press, 2009
12. ↑ Fisher, RA, Tippett, LHC, 1928. Limiting forms of the frequency distribution of the largest or smallest member of a sample. In: Mathematical Proceedings of the Cambridge Philosophical Society. Vol. 24. Cambridge Univ. Press, pp. 180-190
13. ↑ ^{a b} McFadden, D., 1973. Conditional logit analysis of qualitative choice behavior.
14. ↑ Goeree, Holt, Palfrey: Quantal Response Equilibrium, Division of the Humanities and Social Sciences, A Motivating Example: Generalized Matching Pennies, pp. 1 ff.
15. ↑ McKelvey, Richard; Palfrey, Thomas (1998), "Quantal Response Equilibria for Extensive Form Games", Experimental Economics Vol. 1, pp. 9-41