Download intrinsic

Smooth Games and
Intrinsic Robustness
Christodoulou and Koutsoupias,
Roughgarden
Slides stolen/modified from Tim
Roughgarden
1
2
Congestion Games
• Agent i has a set of strategies, Si, each
strategy s in Si is a set of resources
• The cost to an agent is the sum of the
costs of the resources r in s used by the
agent when choosing s
• The cost of a resource is a function of the
number of agents using the resource
fr(# agents)
3
Price of Anarchy
Price of anarchy: [Koutsoupias/Papadimitriou 99]
quantify inefficiency w.r.t some objective
function.
– e.g., Nash equilibrium: an outcome such that no
player better off by switching strategies
Definition: price of anarchy (POA) of a game
(w.r.t. some objective function):
equilibrium objective fn value
the closer to 1
the better
optimal obj fn value
4
Network w/2 players:
2x
s
0
5
12
5x
t
5
Def: the cost C(f) of flow f = sum of all costs
incurred by traffic (avg cost × traffic rate)
x
s
1
½
½
t
s
t
Cost = ½•½ +½•1 = ¾
Formally: if cP(f) = sum of costs of edges of P
(w.r.t. the flow f), then:
C(f) = P fP • cP(f)
6
Def: linear cost fn is of form ce(x)=aex+be
7
Nash Equilibrium:
2x
s
0
5
12
5x
cost = 14+14 = 28
To Minimize Cost:
2x
t
12
0
s
5
t
5x
cost = 14+10 = 24
Price of anarchy = 28/24 = 7/6.
• if multiple equilibria exist, look at the worst one
8
Theorem: [Roughgarden/Tardos 00] for every nonatomic flow network with linear cost fns:
cost of non- ≤
4/3 ×
atomic Nash flow
cost of
opt flow
i.e., price of anarchy non atomic flow ≤ 4/3 in the
linear latency case.
9
Abstract Setup
• n players, each picks a strategy si
• player i incurs a cost Ci(s)
Important Assumption: objective function is
cost(s) := i Ci(s)
Key Definition: A game is (λ,μ)-smooth if, for
every pair s,s* outcomes (λ > 0; μ < 1):
i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)
10
Smooth => POA Bound
Next: “canonical” way to upper bound POA
(via a smoothness argument).
• notation: s = a Nash eq; s* = optimal
Assuming (λ,μ)-smooth:
cost(s) = i Ci(s)
[defn of cost]
≤ i Ci(s*i,s-i)
[s a Nash eq]
≤ λ●cost(s*) + μ●cost(s)
[(*)]
Then: POA (of pure Nash eq) ≤ λ/(1-μ).
11
“Robust” POA
Best (λ,μ)-smoothness parameters:
cost(s) = i Ci(s)
≤ i Ci(s*i,s-i)
≤ λ●cost(s*) + μ●cost(s)
Minimizing: λ/(1-μ).
12
Congestion games with affine
cost functions are (5/3,1/3)smooth
• Claim: For all non-negative integers y, z :
5 2 1 2
y(z + 1) · y + z :
3
3
13
Thus,
y(z + 1)
·
) ay(z + 1) + by
·
5 2 1 2
a,b ≥0
y + z
3
3
5
1
(ay2 + by) + (az2 + bz)
3
3
Let s, s¤ be any two vect ors of st rat egies in a congest ion game,
wit h loads x and x ¤ ,
in (s¤i ; s¡ i ) t he number of users of e is · x e + 1, we have
¤
y
=
x
;
e
k
X
X
¤
Ci (si ; s¡ i ) ·
(ae (x e + 1) + be )x ¤e
i= 1
e2 E
X
·
e2 E
=
 POA  5 / 2
z = xe
X
5
¤
¤
(ae x e + be )x e +
3
e2 E
1
(ae x e + be )x e
3
5
1
C(s¤ ) + C(s):
3
3
14
Why Is Smoothness Stronger?
Key point: to derive POA bound, only needed
i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)
to hold in special case where s = a Nash eq
and s* = optimal.
Smoothness: requires (*) for every pair s,s*
outcomes.
– even if s is not a pure Nash equilibrium
15
The Need for Robustness
Meaning of a POA bound: if the game is at an
equilibrium, then outcome is near-optimal.
16
The Need for Robustness
Meaning of a POA bound: if the game is at an
equilibrium, then outcome is near-optimal.
Problem: what if can’t reach equilibrium?
• (pure) equilibrium might not exist
• might be hard to compute, even centrally
– [Fabrikant/Papadimitriou/Talwar], [Daskalakis/
Goldberg/Papadimitriou], [Chen/Deng/Teng], etc.
• might be hard to learn in a distributed way
Worry: are POA bounds “meaningless”?
17
Robust POA Bounds
High-Level Goal: worst-case bounds that
apply even to non-equilibrium outcomes!
• best-response dynamics, pre-convergence
– [Mirrokni/Vetta 04], [Goemans/Mirrokni/Vetta 05],
[Awerbuch/Azar/Epstein/Mirrokni/Skopalik 08]
• correlated equilibria
– [Christodoulou/Koutsoupias 05]
• coarse correlated equilibria aka “price of
total anarchy” aka “no-regret players”
– [Blum/Even-Dar/Ligett 06],
[Blum/Hajiaghayi/Ligett/Roth 08]
18
Lots of previous work uses
smoothness Bounds
• atomic (unweighted) selfish routing
[Awerbuch/Azar/Epstein 05], [Christodoulou/Koutsoupias 05],
[Aland/Dumrauf/Gairing/Monien/Schoppmann 06], [Roughgarden 09]
• nonatomic selfish routing
[Roughgarden/Tardos 00],[Perakis 04] [Correa/Schulz/Stier Moses 05]
• weighted congestion games
[Aland/Dumrauf/Gairing/Monien/Schoppmann 06],
[Bhawalkar/Gairing/Roughgarden 10]
• submodular maximization games
[Vetta 02], [Marden/Roughgarden 10]
• coordination mechanisms
[Cole/Gkatzelis/Mirrokni 10]
19
Beyond Pure Nash Equilibria
(Static)
Mixed: ¾= ¾1 £ ¾2 £ ¢¢¢£ ¾k
For all s; s0i :
Es» ¾[Ci (s)] · Es¡ i » ¾¡ i [Ci (s0i; s¡ i )]
= ¾1 £ ¾2 £ ¢¢¢£ ¾k
Correlated: ¾6
For all s; s0i : Es» ¾[Ci (s)jsi ] · Es» ¾[Ci (s0i; s¡ i )jsi ]
Coarse Correlated:
For all s; s0i :
Es» ¾[Ci (s)] · E s» ¾[Ci (s0i; s¡ i )]
CCE
correlated eq
mixed Nash
pure
Nash
20
Beyond Nash Equilibria
(non-Static)
Definition: a sequence s1,s2,...,sT of
outcomes is no-regret if:
• for each player i, each
fixed action qi:
– average cost player i incurs
over sequence no worse than
playing action qi every time
no-regret
correlated eq
mixed Nash
pure
Nash
– if every player uses e.g. “multiplicative weights”
then get o(1) regret in poly-time
– empirical distribution = "coarse correlated eq"
21
An Out-of-Equilibrium Bound
Theorem: [Roughgarden STOC 09]
in a (λ,μ)-smooth game, average cost of
every no-regret sequence at most
[λ/(1-μ)] x cost of optimal outcome.
(the same bound we proved for pure Nash equilibria)
22
Smooth => No-Regret Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st)
[defn of cost]
23
Smooth => No-Regret Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st)
= t i [Ci(s*i,st-i) + ∆i,t]
[defn of cost]
[∆i,t:= Ci(st)- Ci(s*i,st-i)]
24
Smooth => No-Regret Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st)
= t i [Ci(s*i,st-i) + ∆i,t]
[defn of cost]
[∆i,t:= Ci(st)- Ci(s*i,st-i)]
≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]
25
Smooth => No-Regret Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st)
= t i [Ci(s*i,st-i) + ∆i,t]
[defn of cost]
[∆i,t:= Ci(st)- Ci(s*i,st-i)]
≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]
No regret: t ∆i,t ≤ 0 for each i.
To finish proof: divide through by T.
26
Intrinsic Robustness
Theorem: [Roughgarden STOC 09] for every set C,
unweighted congestion games with cost
functions restricted to C are tight:
maximum [pure POA] = minimum [λ/(1-μ)]
congestion games
w/cost functions in C
(λ ,μ): all such games
are (λ ,μ)-smooth
27
Intrinsic Robustness
Theorem: [Roughgarden STOC 09] for every set C,
unweighted congestion games with cost
functions restricted to C are tight:
maximum [pure POA] = minimum [λ/(1-μ)]
congestion games
w/cost functions in C
(λ ,μ): all such games
are (λ ,μ)-smooth
• weighted congestion games [Bhawalkar/
Gairing/Roughgarden ESA 10] and submodular
maximization games [Marden/Roughgarden CDC
10] are also tight in this sense
28