Download A Projection Framework for Near

A Projection Framework for NearPotential Polynomial Games
IEEE CDC Maui, December 13th 2012
Nikolai Matni ([email protected])
Control and Dynamical Systems,
California Institute of Technology
Motivation – Potential Games
• Informal definition: local actions
have predictable global
consequences.
• Nice properties
– Pure-strategy Nash Equilibria (NE)
– Simple dynamics converge to these NE
• Applications to distributed control
– Marden, Arslan & Shamma 2010
– Candogan, Menache, Ozdaglar
& Parrilo 2009
– Li & Marden, 2011
Motivation – Polynomial Games
• Would like to consider general class of continuous games
– Finite players, continuous action sets.
• Why?
– Goal is control: most systems of interest are analog.
– Quantization leads to tradeoffs in granularity, performance and
problem dimension.
• Why not?
– Potentially intractable to analyze (Parrilo 2006, Stein et al. 2006 for recent
results).
– Can lead to infinite dimensional optimization problems.
• Solution?
– Restrict ourselves to polynomial cost functions and use Sum Of
Squares (SOS) methods.
Motivation – Near Potential Games
• O. Candogan, A. Ozdalgar, P.A. Parrilo, A Projection
Framework for Near-Potential Games, CDC 2010 (and
subsequent work)
• Basic idea: if a game is “close” to being a potential game, it
behaves “almost as well.”
• Projection Framework – finite dimensional case
– Potential games form a subspace.
– Project onto this framework to find closest potential game.
– If distance from subspace is small, original game inherits many
nice properties.
• Goal: Extend these ideas to polynomial games.
Outline
• Motivation
– Potential games
– Polynomial games
– Near-Potential games
• Preliminaries
– Game Theory
– Algebraic Geometry/Sum of Squares (SOS)
• Projection Framework
• Properties
– Static
– Dynamic
• Example
• Conclusions and Future work
Outline
• Motivation
– Potential games
– Polynomial games
– Near-Potential games
• Preliminaries
– Game Theory
– Algebraic Geometry/Sum of Squares (SOS)
• Projection Framework
• Properties
– Static
– Dynamic
• Example
• Conclusions and Future work
Prelims – Polynomial Game
• A polynomial game
–
–
–
•
is given by:
A finite player set
Strategy spaces
, where
Polynomial utility functions
,
A polynomial game is:
–
–
–
–
Continuous if for all n,
is a closed interval of the real
line
Discrete if for all n,
Mixed if some strategy sets are continuous, and some
are discrete.
Assume w.l.o.g.
Prelims – Potential Games
• A polynomial game G is a polynomial potential
game if there exists a polynomial potential
function
such that, for every
player n, and every
• Algebraic characterization (Monderer, Shapley
’96): A continuous game is a potential game iff
Prelims – Misc. Game Theory
• A strategy
is an approximate Nash (or
ε) Equilibrium if, for all n, we have that
Prelims – SOS and p(x)≥0
• Definition: a real polynomial p(x) admits a Sum Of Squares
(SOS) decomposition if
• Why SOS?
– Determining if p(x)≥0, is in general, NP-hard
– Determining if p(x) is SOS tested through SDP
• Lemma [SOS relaxation]:
If there exist SOS polynomials
then
such that
Outline
• Motivation
– Potential games
– Polynomial games
– Near-Potential games
• Preliminaries
– Game Theory
– Algebraic Geometry/Sum of Squares (SOS)
• Projection Framework
• Properties
– Static
– Dynamic
• Example
• Conclusions and Future work
Projection Framework – MPD & MDD
• Need a notion of distance in the space of games
• Candogan et al. introduced Maximum Pairwise
Distance (MPD)
• Use the continuity of polynomials to define Maximum
Differential Difference (MDD)
• Both capture how different two games are in terms of
utility improvements due to unilateral deviations
Projection Framework
• Task: Given a polynomial game
a nearby potential polynomial game
• Formulate as an optimization problem:
• Constraint ensures we get a Potential Game
• Objective function minimizes MDD.
• Intractable!
, find
Projection Framework – Convexify
• Step 1: rewrite constraint in terms of algebraic
characterization
• Step 2: introduce slack variable γ
Projection Framework – Convexify
• Step 3: apply Lemma [SOS relaxation]
• This is a finite dimensional SOS program,
solvable in polynomial time. It yields a
polynomial potential game satisfying
Projection Framework - Extensions
• Can extend this idea to mixed/discrete games
• Lemma [MPD]:
If
, then
• Continuous Relaxations: For a mixed or discrete game,
set all strategy sets to [-1,1]
– Apply previous SOS program and Lemma [MPD] to mixed
games or discrete games with
– Allows us to apply algebraic characterization, which can
reduce number of constraints from O(
) to O(N)
Outline
• Motivation
– Potential games
– Polynomial games
– Near-Potential games
• Preliminaries
– Game Theory
– Algebraic Geometry/Sum of Squares (SOS)
• Projection Framework
• Properties
– Static
– Dynamic
• Example
• Conclusions and Future work
Properties – Static
• Let and be such that
Then for every ε1-equilibrium y of
ε-equilibrium of , where
.
, z(y) is an
• For continuous games, D=0, z(y)=y, and local
maxima of P are pure (ε=0) NE.
Properties – Static
• Let and be such that
Then for every ε1-equilibrium y of
ε-equilibrium of , where
.
, z(y) is an
• For continuous games, D=0, z(y)=y, and local
maxima of P are pure (ε=0) NE.
Properties – Dynamic
• Definition: ε-better response dynamics
– Round robin updates
– Player updates only to improve utility by at least ε
– Otherwise does not update
• Suppose there exists such that
Then, under ε-better response dynamics, after
a finite number of iterations, dynamics will be
confined to the ε-equilibria set of , for
arbitrary.
Outline
• Motivation
– Potential games
– Polynomial games
– Near-Potential games
• Preliminaries
– Game Theory
– Algebraic Geometry/Sum of Squares (SOS)
• Projection Framework
• Properties
– Static
– Dynamic
• Example
• Conclusions and Future work
Example – Distributed Power
• Consider the N player game
defined by
–
–
–
• Distributed power minimization interpretation
Example – Distributed Power
• Run through projection framework to find
nearby potential game
:
satisfying
Example – Distributed Power
• Potential function concave – can compute
global maximum to identify .2-equilibria of G
• Alternatively, can run .2-better response
dynamics to converge to a .2-equilibria of G.
• Quantify performance through cost function
Example – Distributed Power
• Compare better-response
(xbr) to centralized (optimal
x*) positions
• Better response
comes within ~20% of
centralized solution
• Completely
decentralized
• Arbitrarily scalable
• Requires no a priori
knowledge of base
station locations
Conclusions & Future Work
• Introduce framework for analyzing polynomial games
– Defined MDD and a tractable projection framework to find
nearby potential games
– Related static and dynamic properties of polynomial games
to those of nearby potential games
– Illustrated these methods on a distributed power problem
• Future work
– Projecting onto weighted polynomial games
– Additional static properties (mixed-equilibria)
– Efficiency notions (price of anarchy, price of stability, etc.)