Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Smooth Games and Intrinsic Robustness Christodoulou and Koutsoupias, Roughgarden Slides stolen/modified from Tim Roughgarden 1 2 Congestion Games • Agent i has a set of strategies, Si, each strategy s in Si is a set of resources • The cost to an agent is the sum of the costs of the resources r in s used by the agent when choosing s • The cost of a resource is a function of the number of agents using the resource fr(# agents) 3 Price of Anarchy Price of anarchy: [Koutsoupias/Papadimitriou 99] quantify inefficiency w.r.t some objective function. – e.g., Nash equilibrium: an outcome such that no player better off by switching strategies Definition: price of anarchy (POA) of a game (w.r.t. some objective function): equilibrium objective fn value the closer to 1 the better optimal obj fn value 4 Network w/2 players: 2x s 0 5 12 5x t 5 Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate) x s 1 ½ ½ t s t Cost = ½•½ +½•1 = ¾ Formally: if cP(f) = sum of costs of edges of P (w.r.t. the flow f), then: C(f) = P fP • cP(f) 6 Def: linear cost fn is of form ce(x)=aex+be 7 Nash Equilibrium: 2x s 0 5 12 5x cost = 14+14 = 28 To Minimize Cost: 2x t 12 0 s 5 t 5x cost = 14+10 = 24 Price of anarchy = 28/24 = 7/6. • if multiple equilibria exist, look at the worst one 8 Theorem: [Roughgarden/Tardos 00] for every nonatomic flow network with linear cost fns: cost of non- ≤ 4/3 × atomic Nash flow cost of opt flow i.e., price of anarchy non atomic flow ≤ 4/3 in the linear latency case. 9 Abstract Setup • n players, each picks a strategy si • player i incurs a cost Ci(s) Important Assumption: objective function is cost(s) := i Ci(s) Key Definition: A game is (λ,μ)-smooth if, for every pair s,s* outcomes (λ > 0; μ < 1): i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s) 10 Smooth => POA Bound Next: “canonical” way to upper bound POA (via a smoothness argument). • notation: s = a Nash eq; s* = optimal Assuming (λ,μ)-smooth: cost(s) = i Ci(s) [defn of cost] ≤ i Ci(s*i,s-i) [s a Nash eq] ≤ λ●cost(s*) + μ●cost(s) [(*)] Then: POA (of pure Nash eq) ≤ λ/(1-μ). 11 “Robust” POA Best (λ,μ)-smoothness parameters: cost(s) = i Ci(s) ≤ i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s) Minimizing: λ/(1-μ). 12 Congestion games with affine cost functions are (5/3,1/3)smooth • Claim: For all non-negative integers y, z : 5 2 1 2 y(z + 1) · y + z : 3 3 13 Thus, y(z + 1) · ) ay(z + 1) + by · 5 2 1 2 a,b ≥0 y + z 3 3 5 1 (ay2 + by) + (az2 + bz) 3 3 Let s, s¤ be any two vect ors of st rat egies in a congest ion game, wit h loads x and x ¤ , in (s¤i ; s¡ i ) t he number of users of e is · x e + 1, we have ¤ y = x ; e k X X ¤ Ci (si ; s¡ i ) · (ae (x e + 1) + be )x ¤e i= 1 e2 E X · e2 E = POA 5 / 2 z = xe X 5 ¤ ¤ (ae x e + be )x e + 3 e2 E 1 (ae x e + be )x e 3 5 1 C(s¤ ) + C(s): 3 3 14 Why Is Smoothness Stronger? Key point: to derive POA bound, only needed i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s) to hold in special case where s = a Nash eq and s* = optimal. Smoothness: requires (*) for every pair s,s* outcomes. – even if s is not a pure Nash equilibrium 15 The Need for Robustness Meaning of a POA bound: if the game is at an equilibrium, then outcome is near-optimal. 16 The Need for Robustness Meaning of a POA bound: if the game is at an equilibrium, then outcome is near-optimal. Problem: what if can’t reach equilibrium? • (pure) equilibrium might not exist • might be hard to compute, even centrally – [Fabrikant/Papadimitriou/Talwar], [Daskalakis/ Goldberg/Papadimitriou], [Chen/Deng/Teng], etc. • might be hard to learn in a distributed way Worry: are POA bounds “meaningless”? 17 Robust POA Bounds High-Level Goal: worst-case bounds that apply even to non-equilibrium outcomes! • best-response dynamics, pre-convergence – [Mirrokni/Vetta 04], [Goemans/Mirrokni/Vetta 05], [Awerbuch/Azar/Epstein/Mirrokni/Skopalik 08] • correlated equilibria – [Christodoulou/Koutsoupias 05] • coarse correlated equilibria aka “price of total anarchy” aka “no-regret players” – [Blum/Even-Dar/Ligett 06], [Blum/Hajiaghayi/Ligett/Roth 08] 18 Lots of previous work uses smoothness Bounds • atomic (unweighted) selfish routing [Awerbuch/Azar/Epstein 05], [Christodoulou/Koutsoupias 05], [Aland/Dumrauf/Gairing/Monien/Schoppmann 06], [Roughgarden 09] • nonatomic selfish routing [Roughgarden/Tardos 00],[Perakis 04] [Correa/Schulz/Stier Moses 05] • weighted congestion games [Aland/Dumrauf/Gairing/Monien/Schoppmann 06], [Bhawalkar/Gairing/Roughgarden 10] • submodular maximization games [Vetta 02], [Marden/Roughgarden 10] • coordination mechanisms [Cole/Gkatzelis/Mirrokni 10] 19 Beyond Pure Nash Equilibria (Static) Mixed: ¾= ¾1 £ ¾2 £ ¢¢¢£ ¾k For all s; s0i : Es» ¾[Ci (s)] · Es¡ i » ¾¡ i [Ci (s0i; s¡ i )] = ¾1 £ ¾2 £ ¢¢¢£ ¾k Correlated: ¾6 For all s; s0i : Es» ¾[Ci (s)jsi ] · Es» ¾[Ci (s0i; s¡ i )jsi ] Coarse Correlated: For all s; s0i : Es» ¾[Ci (s)] · E s» ¾[Ci (s0i; s¡ i )] CCE correlated eq mixed Nash pure Nash 20 Beyond Nash Equilibria (non-Static) Definition: a sequence s1,s2,...,sT of outcomes is no-regret if: • for each player i, each fixed action qi: – average cost player i incurs over sequence no worse than playing action qi every time no-regret correlated eq mixed Nash pure Nash – if every player uses e.g. “multiplicative weights” then get o(1) regret in poly-time – empirical distribution = "coarse correlated eq" 21 An Out-of-Equilibrium Bound Theorem: [Roughgarden STOC 09] in a (λ,μ)-smooth game, average cost of every no-regret sequence at most [λ/(1-μ)] x cost of optimal outcome. (the same bound we proved for pure Nash equilibria) 22 Smooth => No-Regret Bound • notation: s1,s2,...,sT = no regret; s* = optimal Assuming (λ,μ)-smooth: t cost(st) = t i Ci(st) [defn of cost] 23 Smooth => No-Regret Bound • notation: s1,s2,...,sT = no regret; s* = optimal Assuming (λ,μ)-smooth: t cost(st) = t i Ci(st) = t i [Ci(s*i,st-i) + ∆i,t] [defn of cost] [∆i,t:= Ci(st)- Ci(s*i,st-i)] 24 Smooth => No-Regret Bound • notation: s1,s2,...,sT = no regret; s* = optimal Assuming (λ,μ)-smooth: t cost(st) = t i Ci(st) = t i [Ci(s*i,st-i) + ∆i,t] [defn of cost] [∆i,t:= Ci(st)- Ci(s*i,st-i)] ≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)] 25 Smooth => No-Regret Bound • notation: s1,s2,...,sT = no regret; s* = optimal Assuming (λ,μ)-smooth: t cost(st) = t i Ci(st) = t i [Ci(s*i,st-i) + ∆i,t] [defn of cost] [∆i,t:= Ci(st)- Ci(s*i,st-i)] ≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)] No regret: t ∆i,t ≤ 0 for each i. To finish proof: divide through by T. 26 Intrinsic Robustness Theorem: [Roughgarden STOC 09] for every set C, unweighted congestion games with cost functions restricted to C are tight: maximum [pure POA] = minimum [λ/(1-μ)] congestion games w/cost functions in C (λ ,μ): all such games are (λ ,μ)-smooth 27 Intrinsic Robustness Theorem: [Roughgarden STOC 09] for every set C, unweighted congestion games with cost functions restricted to C are tight: maximum [pure POA] = minimum [λ/(1-μ)] congestion games w/cost functions in C (λ ,μ): all such games are (λ ,μ)-smooth • weighted congestion games [Bhawalkar/ Gairing/Roughgarden ESA 10] and submodular maximization games [Marden/Roughgarden CDC 10] are also tight in this sense 28