Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Recap Perfect-Information Extensive-Form Games Subgame Perfection Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12 February 15, 2007 Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 1 Recap Perfect-Information Extensive-Form Games Subgame Perfection Lecture Overview Recap Perfect-Information Extensive-Form Games Subgame Perfection Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 2 Recap Perfect-Information Extensive-Form Games Subgame Perfection Example: the sharing game q1 HH 2–0 2 q A no AA yes q (0,0) A Aq (2,0) Extensive Form Games and Subgame Perfection 1–1 HH H q2 A no AA yes A q Aq (0,0) (1,1) 0–2 HH H HHq 2 A no AA yes A Aq q (0,0) (0,2) ISCI 330 Lecture 12, Slide 3 Recap Perfect-Information Extensive-Form Games Subgame Perfection Example: the sharing game q1 HH 2–0 2 q A no AA yes q (0,0) A Aq (2,0) 1–1 HH H q2 A no AA yes A q Aq (0,0) (1,1) 0–2 HH H HHq 2 A no AA yes A Aq q (0,0) (0,2) Play as a fun game, dividing 100 dollar coins. (Play each partner only once.) Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 3 Recap Perfect-Information Extensive-Form Games Subgame Perfection Lecture Overview Recap Perfect-Information Extensive-Form Games Subgame Perfection Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 4 Recap Perfect-Information Extensive-Form Games Subgame Perfection Pure Strategies I In the sharing game (splitting 2 coins) how many pure strategies does each player have? Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 5 Recap Perfect-Information Extensive-Form Games Subgame Perfection Pure Strategies I In the sharing game (splitting 2 coins) how many pure strategies does each player have? I player 1: 3; player 2: 8 Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 5 Recap Perfect-Information Extensive-Form Games Subgame Perfection Pure Strategies I In the sharing game (splitting 2 coins) how many pure strategies does each player have? I I player 1: 3; player 2: 8 Overall, a pure strategy for a player in a perfect-information game is a complete specification of which deterministic action to take at every node belonging to that player. Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 5 Recap Perfect-Information Extensive-Form Games Subgame Perfection Pure Strategies I In the sharing game (splitting 2 coins) how many pure strategies does each player have? I player 1: 3; player 2: 8 I Overall, a pure strategy for a player in a perfect-information game is a complete specification of which deterministic action to take at every node belonging to that player. I Can think of a strategy as a complete set of instructions for a proxy who will play for the player in their abscence Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 5 Recap at each Perfect-Information Extensive-Form Perfection choice node, regardless of whether or not itGames is possible to reach thatSubgame node given the other choice nodes. In the Sharing game above the situation is straightforward— player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. Pure Strategies Example 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) Figure 5.2 H (1,0) A perfect-information game in extensive form. What are the pure strategies for player 2? In order to define a complete strategy for this game, each of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies of the players as follows. S1 = {(A, G), (A, H), (B, G), (B, H)} S2 = {(C, E), (C, F ), (D, E), (D, F )} It isForm important toSubgame note that we have to include the strategies (A, G) and even Extensive Games and Perfection ISCI(A, 330H), Lecture 12, Slide 6 Recap at each Perfect-Information Extensive-Form Perfection choice node, regardless of whether or not itGames is possible to reach thatSubgame node given the other choice nodes. In the Sharing game above the situation is straightforward— player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. Pure Strategies Example 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) Figure 5.2 H (1,0) A perfect-information game in extensive form. What are the pure strategies for player 2? InIorder complete game, S2 to =define {(C,aE); (C, Fstrategy ); (D, for E);this (D, F )}each of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies of the players as follows. S1 = {(A, G), (A, H), (B, G), (B, H)} S2 = {(C, E), (C, F ), (D, E), (D, F )} It isForm important toSubgame note that we have to include the strategies (A, G) and even Extensive Games and Perfection ISCI(A, 330H), Lecture 12, Slide 6 Recap at each Perfect-Information Extensive-Form Perfection choice node, regardless of whether or not itGames is possible to reach thatSubgame node given the other choice nodes. In the Sharing game above the situation is straightforward— player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. Pure Strategies Example 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) Figure 5.2 H (1,0) A perfect-information game in extensive form. What are the pure strategies for player 2? InIorder complete game, S2 to =define {(C,aE); (C, Fstrategy ); (D, for E);this (D, F )}each of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies What are the pure strategies for player 1? of the players as follows. S1 = {(A, G), (A, H), (B, G), (B, H)} S2 = {(C, E), (C, F ), (D, E), (D, F )} It isForm important toSubgame note that we have to include the strategies (A, G) and even Extensive Games and Perfection ISCI(A, 330H), Lecture 12, Slide 6 Recap at each Perfect-Information Extensive-Form Perfection choice node, regardless of whether or not itGames is possible to reach thatSubgame node given the other choice nodes. In the Sharing game above the situation is straightforward— player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. Pure Strategies Example 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) Figure 5.2 H (1,0) A perfect-information game in extensive form. What are the pure strategies for player 2? InIorder complete game, S2 to =define {(C,aE); (C, Fstrategy ); (D, for E);this (D, F )}each of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies What are the pure strategies for player 1? of the players as follows. I S1 = {(B, G); (B, H), (A, G), (A, H)} S1 = {(A, G), (A, H), (B, G), (B, H)} I This is true even though, conditional on taking A, the choice S2 = {(C, E), (C, F ), (D, E), (D, F )} between G and H will never have to be made It isForm important toSubgame note that we have Extensive Games and Perfection to include the strategies (A, G) and even ISCI(A, 330H), Lecture 12, Slide 6 Recap Perfect-Information Extensive-Form Games Subgame Perfection Nash Equilibria Given our new definition of pure strategy, we are able to reuse our old definitions of: I mixed strategies I best response I Nash equilibrium Theorem Every perfect information game in extensive form has a PSNE This is easy to see, since the players move sequentially. Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 7 o q ,0) yes no Recap A A q (0,0) Aq yes no yes Games APerfect-InformationExtensive-Form A A A Aq q Aq (1,1) (0,0) (0,2) Subgame Perfection (2,0) Induced Normal Form Figure 5.1 The Sharing game. at the definition contains a subtlety. An agent’s strategy requires a decision ice node, regardless of whether or not it is possible to reach that node given hoice nodes. In the situationto is straightforward— I the InSharing fact, game the above connection the normal form is even tighter s three pure strategies, and player 2 has eight (why?). But now consider the I we can “convert” an extensive-form game into normal form n in Figure 5.2. 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) Figure 5.2 H (1,0) A perfect-information game in extensive form. o define a complete strategy for this game, each of the players must choose each of his two choice nodes. Thus we can enumerate the pure strategies rs as follows. A, G), (A, H), (B,Games G), (B, Extensive Form andH)} Subgame Perfection ISCI 330 Lecture 12, Slide 8 o q ,0) yes no Recap A A q (0,0) Aq yes no yes Games APerfect-InformationExtensive-Form A A A Aq q Aq (1,1) (0,0) (0,2) Subgame Perfection (2,0) Induced Normal Form Figure 5.1 The Sharing game. at the definition contains a subtlety. An agent’s strategy requires a decision ice node, regardless of whether or not it is possible to reach that node given hoice nodes. In the situationto is straightforward— I the InSharing fact, game the above connection the normal form is even tighter s three pure strategies, and player 2 has eight (why?). But now consider the I we can “convert” an extensive-form game into normal form n in Figure 5.2. 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) Figure 5.2 H AG AH BG BH CE 3, 8 3, 8 5, 5 5, 5 CF 3, 8 3, 8 2, 10 1, 0 DE 8, 3 8, 3 5, 5 5, 5 DF 8, 3 8, 3 2, 10 1, 0 (1,0) A perfect-information game in extensive form. o define a complete strategy for this game, each of the players must choose each of his two choice nodes. Thus we can enumerate the pure strategies rs as follows. A, G), (A, H), (B,Games G), (B, Extensive Form andH)} Subgame Perfection ISCI 330 Lecture 12, Slide 8 q ,0) Aq (2,0) q (0,0) Recap Aq (1,1) q (0,0) Aq (0,2) Perfect-Information Extensive-Form Games Subgame Perfection 5.1 The Sharing game. InducedFigure Normal Form at the definition contains a subtlety. An agent’s strategy requires a decision ice node, regardless of whether or not it is possible to reach that node given hoice nodes. In the situationto is straightforward— I the InSharing fact, game the above connection the normal form is even tighter s three pure strategies, and player 2 has eight (why?). But now consider the I we can “convert” an extensive-form game into normal form n in Figure 5.2. 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) H AG AH BG BH CE 3, 8 3, 8 5, 5 5, 5 CF 3, 8 3, 8 2, 10 1, 0 DE 8, 3 8, 3 5, 5 5, 5 DF 8, 3 8, 3 2, 10 1, 0 (1,0) this illustrates the lack of compactness of the normal form Figure 5.2 I A perfect-information game in extensive form. I games aren’t always small o define a complete strategy for this game, each of thethis players must choose each of his two choice enumerate the 16 purepayoff strategiespairs I nodes. even Thus herewewecanwrite down rs as follows. instead of 5 A, G), (A, H), (B, G), (B, H)} C, E), (C, F ), (D, E), (D, F )} Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 8 ,0) (2,0) Recap (0,0) (1,1) (0,0) Extensive-Form (0,2) Games Perfect-Information Subgame Perfection Figure 5.1 The Sharing game. Induced Normal Form at the definition contains a subtlety. An agent’s strategy requires a decision ice node, regardless of whether or not it is possible to reach that node given hoice nodes. In the situationto is straightforward— I the InSharing fact, game the above connection the normal form is even tighter s three pure strategies, and player 2 has eight (why?). But now consider the I we can “convert” an extensive-form game into normal form n in Figure 5.2. 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) H AG AH BG BH CE 3, 8 3, 8 5, 5 5, 5 CF 3, 8 3, 8 2, 10 1, 0 DE 8, 3 8, 3 5, 5 5, 5 DF 8, 3 8, 3 2, 10 1, 0 (1,0) while we can write any extensive-form game as a NF, we can’t dostrategy the reverse. o define a complete for this game, each of the players must choose Figure 5.2 I A perfect-information game in extensive form. each of his two choice Thus we can enumerate pure strategies I nodes. e.g., matching pennies the cannot be written rs as follows. as a perfect-information extensive form game A, G), (A, H), (B, G), (B, H)} C, E), (C, F ), (D, E), (D, F )} Form andinclude Subgame ant Extensive to note that weGames have to thePerfection strategies (A, G) and (A, H), even ISCI 330 Lecture 12, Slide 8 ,0) (2,0) Recap (0,0) (1,1) (0,0) Extensive-Form (0,2) Games Perfect-Information Subgame Perfection Figure 5.1 The Sharing game. Induced Normal Form at the definition contains a subtlety. An agent’s strategy requires a decision ice node, regardless of whether or not it is possible to reach that node given hoice nodes. In the situationto is straightforward— I the InSharing fact, game the above connection the normal form is even tighter s three pure strategies, and player 2 has eight (why?). But now consider the I we can “convert” an extensive-form game into normal form n in Figure 5.2. 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) H AG AH BG BH CE 3, 8 3, 8 5, 5 5, 5 CF 3, 8 3, 8 2, 10 1, 0 DE 8, 3 8, 3 5, 5 5, 5 DF 8, 3 8, 3 2, 10 1, 0 (1,0) What are the (three) pure-strategy equilibria? Figure 5.2 I A perfect-information game in extensive form. o define a complete strategy for this game, each of the players must choose each of his two choice nodes. Thus we can enumerate the pure strategies rs as follows. A, G), (A, H), (B, G), (B, H)} C, E), (C, F ), (D, E), (D, F )} Form andinclude Subgame ant Extensive to note that weGames have to thePerfection strategies (A, G) and (A, H), even ISCI 330 Lecture 12, Slide 8 ,0) (2,0) Recap (0,0) (1,1) (0,0) Extensive-Form (0,2) Games Perfect-Information Subgame Perfection Figure 5.1 The Sharing game. Induced Normal Form at the definition contains a subtlety. An agent’s strategy requires a decision ice node, regardless of whether or not it is possible to reach that node given hoice nodes. In the situationto is straightforward— I the InSharing fact, game the above connection the normal form is even tighter s three pure strategies, and player 2 has eight (why?). But now consider the I we can “convert” an extensive-form game into normal form n in Figure 5.2. 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) H AG AH BG BH CE 3, 8 3, 8 5, 5 5, 5 CF 3, 8 3, 8 2, 10 1, 0 DE 8, 3 8, 3 5, 5 5, 5 DF 8, 3 8, 3 2, 10 1, 0 (1,0) What are the (three) pure-strategy equilibria? Figure 5.2 I A perfect-information game in extensive form. I (A,for o define a complete strategy this(C, game, G), F each ) of the players must choose each of his two choice Thus(C, we F can) enumerate the pure strategies I nodes. (A, H), rs as follows. I (B, H), (C, E) A, G), (A, H), (B, G), (B, H)} C, E), (C, F ), (D, E), (D, F )} Form andinclude Subgame ant Extensive to note that weGames have to thePerfection strategies (A, G) and (A, H), even ISCI 330 Lecture 12, Slide 8 ,0) (2,0) Recap (0,0) (1,1) (0,0) Extensive-Form (0,2) Games Perfect-Information Subgame Perfection Figure 5.1 The Sharing game. Induced Normal Form at the definition contains a subtlety. An agent’s strategy requires a decision ice node, regardless of whether or not it is possible to reach that node given hoice nodes. In the situationto is straightforward— I the InSharing fact, game the above connection the normal form is even tighter s three pure strategies, and player 2 has eight (why?). But now consider the I we can “convert” an extensive-form game into normal form n in Figure 5.2. 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) H AG AH BG BH CE 3, 8 3, 8 5, 5 5, 5 CF 3, 8 3, 8 2, 10 1, 0 DE 8, 3 8, 3 5, 5 5, 5 DF 8, 3 8, 3 2, 10 1, 0 (1,0) What are the (three) pure-strategy equilibria? Figure 5.2 I A perfect-information game in extensive form. I (A,for o define a complete strategy this(C, game, G), F each ) of the players must choose each of his two choice Thus(C, we F can) enumerate the pure strategies I nodes. (A, H), rs as follows. I (B, H), (C, E) A, G), (A, H), (B, G), (B, H)} C, E), (C, F ), (D, E), (D, F )} Form andinclude Subgame ant Extensive to note that weGames have to thePerfection strategies (A, G) and (A, H), even ISCI 330 Lecture 12, Slide 8 Recap Perfect-Information Extensive-Form Games Subgame Perfection Lecture Overview Recap Perfect-Information Extensive-Form Games Subgame Perfection Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 9 Recap Perfect-Information Extensive-Form Games Notice that the definition contains a subtlety. An agent’s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. In the Sharing game above the situation is straightforward— player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. Subgame Perfection Subgame Perfection 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) Figure 5.2 I H (1,0) A perfect-information game in extensive form. order to define a complete strategy for this game, eachwith of the players choose There’sInsomething intuitively wrong themust equilibrium an action at each of his two choice nodes. Thus we can enumerate the pure strategies (B, H), (C, E) of the players as follows. I Why player S1 =would {(A, G), (A, H), (B,1 G),ever (B, H)}choose to play H if he got to the second choice S2 = {(C, E), (C, Fnode? ), (D, E), (D, F )} Iimportant It is note G that dominates we have to include strategies Aftertoall, Hthefor him(A, G) and (A, H), even though once A is chosen the G-versus-H choice is moot. The definition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfectinformation game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form image of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are Multi Agent Systems, draft of September 19, 2006 Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 10 Recap Perfect-Information Extensive-Form Games Notice that the definition contains a subtlety. An agent’s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. In the Sharing game above the situation is straightforward— player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. Subgame Perfection Subgame Perfection 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) Figure 5.2 I H (1,0) A perfect-information game in extensive form. order to define a complete strategy for this game, eachwith of the players choose There’sInsomething intuitively wrong themust equilibrium an action at each of his two choice nodes. Thus we can enumerate the pure strategies (B, H), (C, E) of the players as follows. I Why player S1 =would {(A, G), (A, H), (B,1 G),ever (B, H)}choose to play H if he got to the second choice S2 = {(C, E), (C, Fnode? ), (D, E), (D, F )} I HeThe does it oftobestthreaten 2, to prevent himas from choosing definition response and player Nash equilibria in this game are exactly they in for normal form games. Indeed, this example illustrates how every perfectFare, and so gets 5 Iimportant It is note G that dominates we have to include strategies Aftertoall, Hthefor him(A, G) and (A, H), even though once A is chosen the G-versus-H choice is moot. information game can be converted to an equivalent normal form game. For example, However, this seems like threat theIperfect-information game of Figure 5.2 can a be non-credible converted into the normal form imageIof the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are If player 1 reached his second decision node, would he really follow through and play Multi Agent Systems, draft ofH? September 19, 2006 Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 10 Recap Perfect-Information Extensive-Form Games Subgame Perfection Formal Definition I Define subgame of G rooted at h: I I Define set of subgames of G: I I I the restriction of G to the descendents of H. subgames of G rooted at nodes in G s is a subgame perfect equilibrium of G iff for any subgame G0 of G, the restriction of s to G0 is a Nash equilibrium of G0 Notes: I I since G is its own subgame, every SPE is a NE. this definition rules out “non-credible threats” Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 11 (0,0) (2,0) (0,0) (1,1) (0,0) Perfect-Information Extensive-Form Games Recap (0,2) Subgame Perfection Figure 5.1 The Sharing game. Back to the Example Notice that the definition contains a subtlety. An agent’s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. In the Sharing game above the situation is straightforward— player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) Figure 5.2 I H (1,0) A perfect-information game in extensive form. In order to define a complete strategy for this game, each of the players must choose Whichan equilibria from thenodes. example subgame perfect? action at each of his two choice Thus we can are enumerate the pure strategies of the players as follows. S1 = {(A, G), (A, H), (B, G), (B, H)} S2 = {(C, E), (C, F ), (D, E), (D, F )} It is important to note that we have to include the strategies (A, G) and (A, H), even though once A is chosen the G-versus-H choice is moot. The definition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfectinformation game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form image of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 12 (0,0) (2,0) (0,0) (1,1) (0,0) Perfect-Information Extensive-Form Games Recap (0,2) Subgame Perfection Figure 5.1 The Sharing game. Back to the Example Notice that the definition contains a subtlety. An agent’s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. In the Sharing game above the situation is straightforward— player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. 1 A B 2 2 C D E (3,8) (8,3) (5,5) F 1 G (2,10) Figure 5.2 I H (1,0) A perfect-information game in extensive form. In order to define a complete strategy for this game, each of the players must choose Whichan equilibria from thenodes. example subgame perfect? action at each of his two choice Thus we can are enumerate the pure strategies I I I of the players as follows. (A,SG), (C, F ) is subgame perfect 1 = {(A, G), (A, H), (B, G), (B, H)} (B,S2H) is an non-credible threat, so (B, H), (C, E) is not = {(C, E), (C, F ), (D, E), (D, F )} subgame It is important perfect to note that we have to include the strategies (A, G) and (A, H), even though onceisA is chosennon-credible, the G-versus-H choiceeven is moot.though H is “off-path” (A, H) also The definition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfectinformation game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form image of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are Extensive Form Games and Subgame Perfection ISCI 330 Lecture 12, Slide 12