Download NauVTslides - Duke University`s Fuqua School of Business

Risk Neutral Equilibria of
Noncooperative Games
Robert Nau
Duke University
April 12, 2013
References
• “Coherent behavior in noncooperative games”
(with K. McCardle, JET 1990)
• “Coherent decision analysis with inseparable probabilities
and utilities” (JRU, 1995)
• “On the geometry of Nash and correlated equilibria”
(with S. G. Canovas & P. Hansen, IJGT 2003)
• “Risk neutral equilibria of noncooperative games”
(Theory and Decision, forthcoming)
• “Efficient correlated equilibria in games of coordination”
(working paper)
• Arbitrage and Rational Choice (manuscript in progress)
Nash vs. Correlated Equilibrium
• What is the most fundamental solution concept for
noncooperative games?
– Nash equilibrium (or a refinement thereof)?
– Rationalizability?
– No, correlated equilibrium.
• A correlated equilibrium is a generalization of Nash
equilibrium in which randomized strategies are permitted
to be correlated.
• More to the point: Nash equilibrium is special case of
correlated equilibrium in which randomization, if any, is
required to be performed independently.
A canonical example: Battle-of-the-Sexes
• Correlation of randomized strategies is most useful in
games where some coordination of strategies is
beneficial or equitable
• Canonical example: the Battle-of-Sexes game
Top
Bottom
Left
2, 1
0, 0
• The obvious solution: flip a coin!
Right
0, 0
1, 2
Nash vs. Correlated Equilibrium
• A Nash equilibrium satisfies a fixed-point condition
analogous to that of a Walrasian equilibrium
• A correlated equilibrium satisfies a no-arbitrage condition
that is a strategic generalization of de Finetti’s fundamental
theorem of probability
Properties of Nash equilibria
• The set of Nash equilibria may be nonconvex and/or
disconnected
• May require randomization with probabilities that are
irrational numbers, even when game payoffs are rational
• Existence proof is based on a fixed point theorem
• Computation can be hard in games with more than a few
players and/or strategies
Properties of correlated equilibria
• The set of correlated equilibria is a convex polytope, like sets of
probabilities that arise elsewhere in decision theory
(incomplete preferences, incomplete markets….)
• Extreme points of the polytope have rational coordinates if the
game payoffs do
• Existence proof only depends on the existence of a stationary
distribution of a Markov chain
• Computation is easy in games of any size: solutions can be
found by linear programming
• The number of extreme points of the polytope can be huge,
but usually only a small number are efficient.
Fundamental theorems of rational choice
•
•
•
•
•
•
•
Subjective probability (de Finetti)
Expected utility (vNM)
Subjective expected utility (Savage)
Decision analysis (Nau)
Asset pricing (Black-Scholes/Merton/Ross….)
(2nd theorem of) Welfare economics (Arrow)
Utilitarianism (Harsanyi)
All of these theorems are duality theorems, provable by separatinghyperplane arguments, in which the primal rationality condition is
no-arbitrage and the dual condition is the existence of a probability
distribution and/or utility function and/or prices and/or weights with
respect to which an action or allocation is optimal.
• Correlated equilibrium, rather than Nash, fits on this list.
The duality theorem for games
• Primal definition of common knowledge:
– The players should be willing to put their money where
their mouth is, i.e., publicly accept bets that reveal the
differences in payoff profiles between their strategies
• Primal definition of strategic rationality:
– The outcome of the game should be jointly coherent, i.e.,
not allow an arbitrage profit to an observer, when the
rules are revealed in this way
• THEOREM (Nau & McCardle 1990): The outcomes of the
game that are jointly coherent are the ones that have positive
support in some correlated equilibrium
How this argument works
• Consider a generic 2x2 game whose “real” rules are:
=
Top
Bottom
Left
a, a
c, c
Right
b, b
d, d
in units of money, with
risk neutral players
• The “revealed” rules (all that can be commonly known under
noncooperative conditions) are encoded in the matrix whose
rows are the payoff vectors of 4 bets on the game’s outcome
that might be offered to an observer:
G=
1TB
1BT
2LR
2RL
TL
a–c
TR
b–d
a – b
b – a
BL
BR
c–a
c – d
d–b
d – c
1TB
1BT
2LR
2RL
TL
a–c
TR
b–d
a – b
b – a
BL
BR
c–a
c – d
d–b
d – c
• “1TB” is the payoff vector of a bet that player 1 should be willing
to accept if she chooses Top in preference to Bottom
• In that case she gets the payoff profile (a, c) rather than
(b, d) as a function of player 2’s choice among (L, R)
• She should choose Top if (a,c) yields a greater expected payoff
than (b,d) at the instant of her move
• In this event, a bet with payoff profile (a-c, b-d) evidently has
non-negative expected value for her.
• She can reveal information about her payoff function by offering
to accept this bet conditional on Top being played
The real game
vs. the revealed game G
• The contents of G (rather than in ) suffice to determine all the
noncooperative equilibria of the game.
• Definition: a correlated equilibrium of the game is a distribution 
that satisfies G  0, i.e., that assigns non-negative expected value
to each of the rows of G
– If used by a mediator to generate recommended strategies, it
would be incentive-compatible for all players to comply
– It is determined merely by linear inequalities
• Definition: a Nash equilibrium is a correlated equilibrium in which
randomization, if any, is independent
But… why require independence? Correlation may be beneficial
and/or it could describe the rational beliefs of an observer who
doesn’t know which equilibrium has been selected
The real game
vs. the revealed game G
• What’s in that is not in G? Information about
externalities: how one player benefits from a change in
another’s strategy, holding her own strategy fixed.
• The missing information is what may give rise to dilemmas
in which strategically rational behavior is not socially
rational
• It also determines which side of the correlated equilibrium
polytope is the efficient frontier.
Geometry of Nash & Correlated Equilibria
• THEOREM (Nau et al. 2003): The Nash equilibria of a game
are all located on the surface of the correlated equilibrium
polytope, i.e., on supporting hyperplanes to it.
• Example: Battle-of-the-sexes
• The set of correlated equilibria is a hexadron, with 5
vertices and 6 faces.
– 2 of the vertices are pure Nash equilibria
– 1 is a mixed Nash equilibrium
– 2 are extremal non-Nash correlated equilibria
The correlated equilibrium polytope of Battle-of-the-Sexes
Top
Left
2, 1
Right
0, 0
Bottom
0, 0
1, 2
• The polytope is determined by a system of linear
inequalities: incentive constraints for not deviating
from the strategy recommended to you by a
possibly-correlated randomization device
• The Nash equilibria are points where the polytope
touches the “saddle” of independent distribuitions
The pure Nash equilibria
(as seen from 2 angles)
The mixed Nash equilibrium
In general Nash equilibria do
not need to be vertices of the
polytope: they can fall in the
middle of edges or lie on
curves within faces of it
The extremal non-Nash
correlated equilibria
The “obvious” coin-flip solution of
battle-of-the-sexes is the midpoint
of the edge connecting TL and BR,
which is neither a Nash equilibrium
nor a vertex of the polytope
• THEOREM: In a 2x2 game with a 3-dimensional correlated
equilibrium polytope (like BOS), the efficient frontier may
only consist of one of the following:
i.
One of the two pure Nash equilibra
ii.
Both of the pure Nash equilibria, and their convex
combinations
iii. Both of the pure Nash equilibria and one of the two
extremal non-Nash correlated equilibria, and their
convex combinations
• The mixed strategy Nash equilibrium is never efficient,
no matter what the externalities are.
• Completely mixed Nash equilibria are generally inefficient in
games with multiple equilibria, because they satisfy the
incentive constraints with equality and hence they are also
equilibria of the game with the opposite payoffs.
• Conjecture: a completely mixed Nash equilibrium can
never be efficient in any game whose correlated
equilibrium polytope is of maximal dimension
(i.e., dimension n-1 in a game with n outcomes),
which is a generalized game of coordination.
• Seems to be empirically true based on numerical
experiments, but a general proof is (so far) elusive
• Example of a game with a unique Nash equilibrium in
completely mixed strategies that is not a vertex of the
correlated equilibrium polytope.
• Similar to a “poker” game devised by Shapley and
discussed by Nash (1951)
• The polytope is 7-dimensional (i.e., full dimension) with
33 vertices
• The Nash equilibrium lies in the middle of an edge and
has irrational coordinates.
• Example: a game with a continuum of inefficient Nash
equilibria lying in a face of the polytope:
• The CE polytope is has 7 vertices (full dimension).
• By manipulating externalities, any of the vertices can be
placed on the efficient frontier.
• The Nash equilibria lie in a face of the polytope along
an open curve connecting two of the vertices.
• The face containing the Nash equilibria can be placed
on the inefficient frontier, but not the efficient frontier.
A unified theory of individual, strategy, and
competitive rationality?
• The solution concept of correlated equilibrium provides a
tight link between subjective probability theory (á la de
Finetti), game theory (á la Aumann), and finance theory (á la
Black-Scholes)
• In all of the fundamental theorems, the subjective
parameters of belief are revealed through betting, and the
rationality criterion is that of no-arbitrage.
• But there’s a catch: the interpretation of the probabilities as
measures of pure belief is complicated by the presence of
risk aversion.
• What if utility for money is state-dependent due to risk
aversion and prior stakes in events?
Risk neutral probabilities
• In financial markets, the probability distributions that
rationalize asset prices are not measures of pure belief.
• They are “risk neutral probabilities” that characterize a “risk
neutral representative agent” but which do NOT reflect the
beliefs of risk averse real agents
• Risk neutral probabilities are interpretable as products of true
probabilities and state-dependent marginal utilities
• The average real agent is risk averse with higher marginal
utilities for money in states where asset prices are low, so her
risk neutral distribution is typically left-shifted compared to her
true probability distribution.
• The mean of the risk neutral distribution of asset returns must
equal the risk free rate of return
Risk neutral equilibria
• How does the presence of risk aversion and large stakes
change the solution of the game whose parameters are made
commonly known through betting?
– The parameters of equilibria become risk neutral
probabilities!
– It also changes the geometry of the set of equilibria
– The polytope of equilibria is distorted in a systematic way,
analogous to the way the risk neutral distribution of an
asset prices is left-shifted.
– The polytope “blows up” to a larger size.
Example: Matching pennies
Top
Bottom
Left
1, 1
1, 1
Right
1, 1
1, 1
• This is a zero-sum game, not a coordination game like Battleof-the-Sexes or Chicken
• The correlated equilibrium polytope consists of a single
point—a mixed Nash equilibrium in which both players use
independent 50/50 randomization when payoffs are in units
of money and both players are risk neutral
The rules-of-the-game
matrix G looks like this:
1TB
1BT
2LR
2RL
TL
1
0
1
0
TR
-1
0
0
-1
BL
0
-1
-1
0
BR
0
1
0
1
…and the unique
Nash/correlated equilibrium
the point in the middle of
the saddle of independent
distributions (the uniform
distribution)
Risk averse players
• Suppose players in such a game are significantly risk averse
(i.e., have small risk tolerances compared to game payoffs)?
• The game is then non-zero-sum in units of utility
• The very idea of a game that is zero-sum when utility for
money is not linear does not really make sense.
• In general, strategic incentives measured in units of utility
cannot be made common knowledge through conversations
in terms of acceptable bets.
• Consider a generic 2x2 game whose monetary payoffs are
Top
Bottom
Left
a, a
c, c
Right
b, b
d, d
• Let u1(.) and u2(.) denote the players’ utility functions
• The rows of the (unobservable) “true” rules-of-the-game
matrix would be bets with payoffs in units of utility:
TL
u1(a) – u1(c)
G*
TR
u1(b) – u1(d)
BL
BR
1TB
u1(c) – u1(a) u1(b) – u1(d)
= 1BT
2LR u1(a) – u1(b)
u1(c) – u1(d)
2RL
u1(b) – u1(a)
u1(d) – u1(c)
… because this is the necessary condition for them to have zero
expected utility w.r.t. the players’ true probabilities
• Let v1(.) and v2(.) denote the players’ marginal utility functions
(i.e., vi is the derivative of ui)
• Then the payoff vectors of acceptable monetary bets that reveal
their strategic incentives (denoted “G*”) looks like this:
1TB
1BT
2LR
2RL
TL
(u1(a) – u1(c))/v1(a)
TR
(u1(c) – u1(a))/v1(a)
(u2(a) – u1(b))/v2(a)
BL
BR
(u1(b) – u1(d))/v1(c)
(u1(b) – u1(d))/v1(c)
(u2(c) – u1(d))/v2(c)
(u2(b) – u1(a))/v2(b)
(u2(d) – u1(c))/v2(c)
• The payoffs of the bets are in units of money, and they are
distorted compared to what their payoffs would have been
under linear utility for money
• The monetary payoff of a bet (e.g., (u1(a) – u1(c))/v1(a)) is the
true-game utility difference divided by the marginal utility that
prevails in that outcome of the game (e.g., v1(a))
Risk neutral equilibria defined
• The appropriate generalization of a correlated equilibrium
under conditions where players are risk averse is a risk neutral
equilibrium
• Definition: a risk neutral equilibrium of the game is a
distribution  that satisfies G*  0, i.e., that assigns nonnegative expected value to each of the rows of G*
• Implementation would require a mediator to use a
randomization device whose inputs include events about
which players might have different true probabilities but
would must have identical risk neutral probabilities, as if they
have already reached equilibrium in a contingent claims
market with assets pegged to those events.
Example: matching pennies with monetary payoffs of 1:
Top
Bottom
Left
1, 1
1, 1
Right
1, 1
1, 1
Let u(x) = 1  exp(x) for both players, with risk aversion  = LN(2).
Then the rules of the game in units of utility are:
Top
Bottom
Left
a, b
b, a
Right
b, a
a, b
where a = 1  ½  0.293 and b = 1  2  0.414.
This is a negative sum game in units of utility.
The local marginal utilities are u’(a) = 0.245 and u’(b) = 0.490
The real rules of the game matrix, G*, whose rows are the
acceptable monetary bets (scaled to maximum of +1):
1TB
1BT
2LR
2RL
TL
1
0
1/2
0
TR
1/2
0
0
1
BL
0
1/2
1
0
BR
0
1
0
1/2
The risk-neutral-equilibrium polytope, determined by the
inequalities G*  0, has 4 vertices, each of which assigns
strictly positive expected value to exactly one of the bets:
Vertex 1
Vertex 2
Vertex 3
Vertex 4
TL
2/15
8/15
4/15
1/15
TR
4/15
1/15
8/15
2/15
BL
1/15
4/15
2/15
8/15
BR
8/15
2/15
1/15
4/15
EV>0?
1BT
1TB
2RL
2LR
Main result
• THEOREM: The set of correlated equilibria of a nontrivial
game with monetary payoffs played by strictly risk
neutral players is a strict subset of the set of risk neutral
equilibria of the same game played by risk averse players.
• The set of risk neutral equilibria of
Matching Pennies is a tetrahedron
with points on both sides of the
saddle of independent distributions.
• The unique Nash/correlated
equilibrium is in its interior.
• From an observer’s perspective, the
players’ reciprocal beliefs are less
precisely determined than in the risk
neutral case.
Conclusions
• The game-theoretic assumption of common knowledge of
payoff functions can be made precise in terms of the language
of betting.
• The assumption of common knowledge of rationality can be
made precise in terms of no-arbitrage.
• When players are risk neutral the solution concept to which
this leads is correlated equilibrium, not Nash equilibrium.
• A Nash equilibrium is a special case of a correlated equilibrium,
not the other way around.
• The “solution” of a game is generally a convex set of correlated
equilibria.
• The constraints that determine equilibria ignore information
about externalities, which is why strategic equilibria are not
necessarily efficient.
Conclusions
• When the players in a game are risk averse, the assumption
that they have common knowledge of each others’ utilities is
very problematic.
• There is no “market” in which assets are priced in units of
utility, and prices that agents are willing to pay for monetary
assets do not reveal their utility functions.
• This paper proposes a method that theoretically could address
this problem, and it leads to the conclusion that in games with
risk averse players:
– The parameters of strategic equilibria are risk neutral
probabilities, as in financial markets with risk averse
investors
– They are less-precisely determined (even for zero-sum
games) than if players were risk averse.