Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
GameTheory -Lecture5 PatrickLoiseau EURECOM Fall2016 1 Lecture3-4recap • DefinedmixedstrategyNashequilibrium • ProvedexistenceofmixedstrategyNashequilibriumin finitegames • Discussedcomputationandinterpretationofmixed strategiesNashequilibrium • Definedanotherconceptofequilibriumfrom evolutionarygametheory àToday:introduceothersolutionconceptsfor simultaneousmovesgames àIntroducesolutionsforsequentialmovesgames 2 Outline • Othersolutionconceptsforsimultaneous moves – Stabilityofequilibrium • Trembling-handperfectequilibrium – Correlatedequilibrium – Minimax theoremandzero-sumgames – ε-Nashequilibrium • Thelenderandborrowergame:introduction andconceptsfromsequentialmoves 3 Outline • Othersolutionconceptsforsimultaneous moves – Stabilityofequilibrium • Trembling-handperfectequilibrium – Correlatedequilibrium – Minimax theoremandzero-sumgames – ε-Nashequilibrium • Thelenderandborrowergame:introduction andconceptsfromsequentialmoves 4 TheLocationModel • Assumewehave2N playersinthisgame(e.g.,N=70) – Playershavetwotypes:tallandshort – ThereareN tallplayersandN shortplayers • Playersarepeoplewhoneedtodecideinwhichtowntolive • Therearetwotowns:EasttownandWesttown – EachtowncanhostnomorethanN players • Assume: – Ifthenumberofpeoplechoosingaparticulartownislargerthanthe towncapacity,thesurpluswillberedistributedrandomly • Game: – Players:2N people – Strategies:EastorWesttown – Payoffs 5 TheLocationModel:payoffs Utilityforplayeri • Theideais: – Ifyouareasmall minority inyourtown yougetapayoffofzero – Ifyouareinlarge majority inyourtown yougetapayoffof½ – Ifyouarewell integrated yougeta payoffof1 1 1/2 0 • Peoplewouldliketolive inmixedtowns,butif theycannot,thenthey prefertoliveinthe majoritytown 35 70 #ofyourtype inyourtown 6 Initialstate • Assumetheinitial pictureisthisone • Whatwillplayersdo? Tallplayer Shortplayer WestTown EastTown 7 Firstiteration • Fortallplayers • There’saminorityof easttown“giants”to beginwith à switchtoWesttown Tallplayer Shortplayer WestTown EastTown • Forshortplayers • There’saminorityof westtown“dwarfs”to beginwith àswitchtoEasttown 8 Seconditeration • Sametrend • Stillafewplayerswho didnotunderstand – Whatistheirpayoff? Tallplayer Shortplayer WestTown EastTown 9 Lastiteration • Peoplegotsegregated • Buttheywouldhave preferredintegratedtowns! – Why?Whathappened? – Peoplethatstartedina minority(eventhough nota“bad”minority) hadincentivestodeviate Tallplayer Shortplayer WestTown EastTown 10 TheLocationModel:Nashequilibria • TwosegregatedNE: – Short,E;Tall,W – Short,W;Tall,E • IsthereanyotherNE? 11 Stabilityofequilibria • Theintegratedequilibriumisnotstable – Ifwemoveawayfromthe50%ratio,evenalittlebit,playershavean incentivetodeviateevenmore – Weendupinoneofthesegregatedequilibrium • Thesegregatedequilibria arestable – Introduceasmallperturbation:playerscomebacktosegregation quickly • NotionofstabilityinPhysics:ifyouintroduceasmallperturbation, youcomebacktotheinitialstate • Tippingpoint: – IntroducedbyGrodzins (WhiteflightsinAmerica) – ExtendedbyShelling(Nobelprizein2005) 12 Trembling-handperfectequilibrium Definition: Trembling-handperfectequilibrium A (mixed)strategyprofilesisatrembling-hand perfectequilibriumifthereexistsasequence s(0),s(1),…offullymixedstrategyprofilesthat convergestowardssandsuchthatforallkand allplayeri,si isabestresponsetos(k)-i. • Fully-mixedstrategy:positiveprobabilityoneach action • Informally:aplayer’sactionsi mustbeBRnot onlytoopponentsequilibriumstrategiess-i but alsotosmallperturbationsofthoses(k)-i. 13 TheLocationModel • Thesegregatedequilibria aretrembling-hand perfect • Theintegratedequilibriumisnottremblinghandperfect 14 Outline • Othersolutionconceptsforsimultaneous moves – Stabilityofequilibrium • Trembling-handperfectequilibrium – Correlatedequilibrium – Minimax theoremandzero-sumgames – ε-Nashequilibrium • Thelenderandborrowergame:introduction andconceptsfromsequentialmoves 15 Example:battleofthesexes Player2 Player1 Opera Soccer Opera Soccer 2,1 0,0 0,0 1,2 • NE:(O,O),(S,S)and((1/3,2/3),(2/3,1/3)) – Themixedequilibriumhaspayoff2/3each • Supposetheplayerscanobservetheoutcomeofafair tosscoinandconditiontheirstrategiesonthis outcome – Newstrategiespossible:Oifhead,Siftails – Payoff1.5each • Thefaircoinactsasacorrelatingdevice 16 Correlatedequilibrium:generalcase • Inthepreviousexample:bothplayersobservethe exactsamesignal(outcomeofthecointossrandom variable) • Generalcase:eachplayerreceivesasignalwhichcan becorrelatedtotherandomvariable(cointoss)andto theotherplayerssignal • Model: – nrandomvariables(oneperplayer) – AjointdistributionoverthenRVs – Naturechoosesaccordingtothejointdistributionand revealstoeachplayeronlyhisRV à AgentcanconditionhisactiontohisRV(hissignal) 17 Correlatedequilibrium:definition Definition: Correlatedequilibrium Acorrelatedequilibriumofthegame(N,(Ai),(ui))is atuple(v,π,σ)where • v=(v1,…,vn)isatupleofrandomvariableswith domains(D1,…,Dn) • πisajointdistributionoverv • σ=(σ1,…,σn)isavectorofmappingsσi:DiàAi suchthatforalli andanymappingσi’:DiàAi, ∑ d∈D1××Dn π (d)u(σ 1 (d1 ),, σ i (di ),, σ n (dn )) ≥ ∑ d∈D1××Dn π (d)u(σ 1 (d1 ),, σ i%(di ),, σ n (dn )) 18 Correlatedvs Nashequilibrium • Thesetofcorrelatedequilibria containsthe setofNashequilibria Theorem: ForeveryNashequilibriumσ*,thereexistsa correlatedequilibrium(v,π,σ)suchthatforeach playeri,thedistributioninducedonAiisσi*. • Proof:constructitwithDi=Ai,independent signals(π(d)=σ*1(d1)x…xσ*n(dn))andidentity mappingsσi 19 Correlatedvs Nashequilibrium(2) • Notallcorrelatedequilibria correspondtoa Nashequilibrium • Example,thecorrelatedequilibriuminthe battle-of-sexgame à Correlatedequilibriumisastrictlyweaker notionthanNE 20 Outline • Othersolutionconceptsforsimultaneous moves – Stabilityofequilibrium • Trembling-handperfectequilibrium – Correlatedequilibrium – Minimax theoremandzero-sumgames – ε-Nashequilibrium • Thelenderandborrowergame:introduction andconceptsfromsequentialmoves 21 Maxmin strategy • Maximize“worst-casepayoff” Definition: Maxmin strategy Themaxmin strategyforplayeri is arg max min ui (si , s−i ) s−i si defender • Example – Attacker:Notattack – Defender:Defend Defend Notdef attacker Attack Notatt -2,1 0,-1 2,-2 0,0 • ThisisnotaNashequilibrium! 22 Maxmin strategy:intuition • Playeri commitstostrategysi (possiblymixed) • Player–i observesi andchooses-i tominimize i’spayoff • Playeri guaranteespayoffatleastequaltothe maxmin value max min ui (si , s−i ) si s−i 23 Twoplayerszero-sumgames • Definition:a2-playerszero-sumgameisagame whereu1(s)=-u2(s)forallstrategyprofiles – Sumofpayoffsconstantequalto0 • Example:Matchingpennies • Defineu(s)=u1(s) – Player1:maximizer – Player2:minimizer heads Player2 heads tails 1,-1 -1,1 Player1 tails -1,1 1,-1 24 Minimax theorem Theorem: Minimax theorem(VonNeumann1928) Foranytwo-playerzero-sumgamewithfiniteaction space: max min u(s1, s2 ) = min max u(s1, s2 ) s1 s2 s2 s1 • Thisquantityiscalledthevalue ofthegame – correspondstothepayoffofplayer1atNE • Maxmin strategiesó NEstrategies • Canbecomputedinpolynomialtime(through linearprogramming) 25 Outline • Othersolutionconceptsforsimultaneous moves – Stabilityofequilibrium • Trembling-handperfectequilibrium – Correlatedequilibrium – Minimax theoremandzero-sumgames – ε-Nashequilibrium • Thelenderandborrowergame:introduction andconceptsfromsequentialmoves 26 ε-Nashequilibrium Definition: ε-Nashequilibrium Forε>0,astrategyprofile(s1*,s2*,…,sN*)isanεNashequilibriumif,foreachplayeri, ui(si*,s-i*)≥ui(si,s-i*)- ε forallsi ≠si* • ItisanapproximateNashequilibrium – Agentsindifferenttosmallgains(couldnotgain morethanε byunilateraldeviation) • ANashequilibriumisanε-Nashequilibrium forallε! 27 Outline • Othersolutionconceptsforsimultaneous moves – Stabilityofequilibrium • Trembling-handperfectequilibrium – Correlatedequilibrium – Minimax theoremandzero-sumgames – ε-Nashequilibrium • Thelenderandborrowergame:introduction andconceptsfromsequentialmoves 28 “CashinaHat”game(1) • Twoplayers,1and2 • Player1strategies:put$0,$1or$3inahat • Then,thehatispassedtoplayer2 • Player2strategies:either“match”(i.e.,add thesameamountofmoneyinthehat)ortake thecash 29 “CashinaHat”game(2) Payoffs: • Player1: $0à $0 $1à ifmatchnetprofit$1,-$1ifnot $3à ifmatchnetprofit$3,-$3ifnot • Player2: Match$1 à Netprofit$1.5 Match$3à Netprofit$2 Takethecashà $inthehat 30 Lender&Borrowergame • The“cashinahat”gameisatoyversionofthe moregeneral“lenderandborrower”game: – Lenders:Banks,VCFirms,… – Borrowers:entrepreneurswithprojectideas • Thelenderhastodecidehowmuchmoneyto invest intheproject • Afterthemoneyhasbeeninvested,theborrower could – Goforwardwiththeprojectandworkhard – Shirk,andruntoMexicowiththemoney 31 Simultaneousvs.SequentialMoves • Whatisdifferentaboutthisgamewrt games studieduntilnow? • Itisasequentialmovegame – Playerchoosesfirst,thenplayer2 • Timingisnotthekey – ThekeyisthatP2observesP1’schoicebefore choosing – AndP1knowsthatthisisgoingtobethecase 32 Extensiveformgames • Ausefulrepresentationofsuchgamesisgame trees alsoknownastheextensiveform – Eachinternalnodeofthetreewillrepresentthe abilityofaplayertomakechoicesatacertain stage,andtheyarecalleddecisionnodes – Leafsofthetreearecalledendnodes and representpayoffstobothplayers • Normalformgamesà matrices • Extensiveformgamesà trees 33 “Cashinahat”representation 2 (0,0) $0 1 $1 2 $3 2 $1 (1,1.5) - $1 (-1,1) $3 (3,2) - $3 (-3,3) Howtoanalyzesuchgame? 34 BackwardInduction • Fundamentalconceptingametheory • Idea:playersthatmoveearlyoninthegameshouldput themselvesintheshoesofotherplayersplayinglater à anticipation • Lookattheendofthetreeandworkbacktowardstheroot – Startwiththelastplayerandchosethestrategiesyielding higherpayoff – Thissimplifiesthetree – Continuewiththebefore-lastplayeranddothesamething – Repeatuntilyougettotheroot 35 BackwardInductioninpractice(1) 2 (0,0) $0 1 $1 2 $3 2 $1 (1,1.5) - $1 (-1,1) $3 (3,2) - $3 (-3,3) 36 BackwardInductioninpractice(2) 2 (0,0) $0 1 $1 2 (1,1.5) $3 2 (-3,3) 37 BackwardInductioninpractice(3) 2 (0,0) $0 1 $1 2 $3 2 $1 (1,1.5) - $1 (-1,1) $3 (3,2) - $3 (-3,3) Outcome: Player1choosestoinvest$1,Player2matches 38 Theproblemwiththe “lendersandborrowers”game • Itisnotadisaster: – Thelenderdoubledhermoney – Theborrowerwasabletogoaheadwithasmallscaleproject andmakesomemoney • But,wewouldhavelikedtoendupinanotherbranch: – Largerprojectfundedwith$3andanoutcomebetterforboth thelenderandtheborrower • Verysimilartoprisoner’sdilemna • Whatpreventsusfromgettingtothislattergoodoutcome? 39 MoralHazard • Oneplayer(theborrower)hasincentivestodothingsthatarenot intheinterestsoftheotherplayer(thelender) – Bygivingatoobigloan,theincentivesfortheborrowerwillbesuch thattheywillnotbealignedwiththeincentivesonthelender – Noticethatmoralhazardhasalsodisadvantagesfortheborrower • Example:Insurancecompaniesoffers“full-risk”policies – Peoplesubscribingforthispoliciesmayhavenoincentivestotake care! – Inpractice,insurancecompaniesforcemetobearsomedeductible costs(“franchise”) • Onepartyhasincentivetotakeariskbecausethecostisfeltby anotherparty • HowcanwesolvetheMoralHazardproblem? 40 Solution(1):Introducelaws • Todaywehavesuchlaws:bankruptcylaws • But,therearelimitstothedegreetowhich borrowerscanbepunished – Theborrowercansay:Ican’trepay,I’mbankrupt – Andhe/she’smoreorlessallowedtohaveafresh start 41 Solution(2):Limits/restrictionson money • Asktheborrowersaconcreteplan(business plan)onhowhe/shewillspendthemoney • Thisboilsdowntochangingtheorderofplay! • Alsofacessomeissues: – Lackofflexibility,whichisthemotivationtobean entrepreneurinthefirstplace! – Problemoftiming:itissometimeshardtopredict up-frontalltheexpensesofaproject 42 Solution(3):Breaktheloanup • Lettheloancomeinsmallinstallments • Ifaborrowerdoeswellonthefirst installment,thelenderwillgiveabigger installmentnexttime • Itissimilartotakingthisone-shotgameand turnitintoarepeatedgame 43 Solution(4):Changecontracttoavoid shirk-- Incentives • Theborrowercouldre-designthepayoffsofthegamein casetheprojectissuccessful 2 2 (0,0) $0 1 $1 2 $3 2 $1 $0 (1,1.5) 1 - $1 (-1,1) $3 (3,2) - $3 (-3,3) (0,0) $1 2 $3 2 $1 (1,1.5) - $1 (-1,1) $3 (1.9,3.1) - $3 (-3,3) • Profitdoesn’tmatchinvestmentbuttheoutcomeisbetter – Sometimesasmallershareofalargerpiecanbe biggerthanalargershareofasmallerpie 44 Absolutepayoffvs ROI • Previousexample:largerabsolutepayoffin thenewgameontheright,butsmallerreturn oninvestment(ROI) • Whichmetric(absolutepayofforROI)should aninvestmentbanklookat? 45 Solution(5):Beyondincentives, collaterals • Theborrowercouldre-designthepayoffsofthe gameincasetheprojectissuccessful – Example:subtracthousefromrunawaypayoffs 2 (0,0) $0 1 $1 2 $1 - $1 $3 2 (1,1.5) (-1,1- HOUSE) $3 (3,2) - $3 (-3,3- HOUSE) – Lowersthepayoffstoborroweratsometreepoints, yetmakestheborrowerbetteroff! 46 Collaterals • Theydohurtaplayerenoughtochange his/herbehavior èLoweringthepayoffsatcertainpointsofthe game,doesnotmeanthataplayerwillbe worseoff!! • Collateralsarepartofalargerbranchcalled commitmentstrategies – Next,anexampleofcommitmentstrategies 47 NormanArmyvs.SaxonArmyGame • Collateralsarepartofalargerbranchcalled commitmentstrategies • Backin1066,WilliamtheConquerorleadan invasionfromNormandyontheSussexbeaches • We’retalkingaboutmilitarystrategy • Sobasicallywehavetwoplayers(thearmies)and thestrategiesavailabletotheplayersare whetherto“fight”or“run” 48 NormanArmyvs.SaxonArmyGame N S N invade fight run N fight run fight run (0,0) (1,2) (2,1) (1,2) Let’sanalyzethegamewith BackwardInduction 49 NormanArmyvs.SaxonArmyGame N S N invade fight run N fight run fight run (0,0) (1,2) (2,1) (1,2) 50 NormanArmyvs.SaxonArmyGame N S N invade fight run (1,2) N (2,1) 51 NormanArmyvs.SaxonArmyGame N S N invade fight run N fight run fight run BackwardInductiontellsus: • Saxonswillfight • Normanswillrunaway (0,0) (1,2) (2,1) (1,2) WhatdidWilliamthe Conquerordo? 52 NormanArmyvs.SaxonArmyGame N S fight run N N Notburn boats run fight run Burnboats S fight fight run N fight N fight (0,0) (1,2) (2,1) (1,2) (0,0) (2,1) 53 NormanArmyvs.SaxonArmyGame N S fight run N run N fight N fight N fight (1,2) (2,1) Notburn boats Burnboats S fight run (0,0) (2,1) 54 NormanArmyvs.SaxonArmyGame S N (1,2) Notburn boats Burnboats S (2,1) 55 NormanArmyvs.SaxonArmyGame N S fight run N N Notburn boats run fight run Burnboats S fight fight run N fight N fight (0,0) (1,2) (2,1) (1,2) (0,0) (2,1) 56 Commitment • Sometimes,gettingridofchoicescanmakeme betteroff! • Commitment: – Feweroptionschangethebehaviorofothers • Theotherplayersmustknow aboutyour commitments – Example:Dr.Strangelovemovie 57