Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Game Theory & Cognitive Radio part A Hamid Mala Presentation Objectives 2/28 1. Basic concepts of game theory 2. Modeling interactive Cognitive Radios as a game 3. Describe how/when game theory applies to cognitive radio. 4. Highlight some valuable game models. Interactive Cognitive Radios Adaptations of one radio can impact adaptations of others Interactive Decisions Difficult to Predict Performance 3/28 Interactive Cognitive Radios Scenario: Distributed SINR maximizing power control in a single cluster. Final state : All nodes transmit at maximum power. (1) the resulting SINRs are unfairly distributed (the closest node will have a far superior SINR to the furthest node) (2) battery life would be greatly shortened. 4/28 Power SINR traditional analysis techniques 5/28 Dynamical systems theory optimization theory contraction mappings Markov chain theory Research in a nutshell 6/28 Applying game theory and game models (potential and supermodular) to the analysis of cognitive radio interactions – Provides a natural method for modeling cognitive radio interactions – Significantly speeds up and simplifies the analysis process – Permits analysis without well defined decision processes Game Theory Definition, Key Concepts opposite color winner Exaple $ = card number of winner 8/28 Same color winner opposite color winner Exaple $ = card number of winner 9/28 Same color winner Exaple Matrix representation Girl’s strategies Boy’s strategies 10/28 (2,-2) (-8,8) (-1,1) (7,-7) Pay-off function Games A game is a model (mathematical representation) of an interactive decision situation. Its purpose is to create a formal framework that captures the relevant information in such a way that is suitable for analysis. Different situations indicate the use of different game models. Normal Form Game Model 1. 2. 3. 11/28 A set of 2 or more players, N A set of actions for each player, Ai A set of utility functions, {ui}, that describe the players’ preferences over the outcome space Nash Equilibrium An action vector from which no player can profitably unilaterally deviate. Definition An action tuple a is a NE if for every i N ui ai , ai ui bi , ai for all bi Ai. 12/28 Friend or Foe Example Friend Friend 500,500 0,1000 Foe 1000,0 0,0 (Friend, Friend)?? 13/28 Foe No (Friend, Foe)?? Yes (Foe, Friend)?? Yes (Foe, Foe)?? Yes Modeling and Analysis Review Modeling a Network as a Game Network Game Nodes Players Power Levels Actions Algorithms Utility Functions Structure of game is taken from the algorithm and the environment void update_power(void) { /*Adjusting power level*/ int k; } [Laboratoire de Radiocommunications et de Traitement du Signal] 15/28 Modeling Review The interactions in a cognitive radio network can be represented by the tuple: <N, A, {ui}, {di},T> Timings: – – – – 16/28 Synchronous Round-robin Random Asynchronous Dynamical System Key Issues in Analysis 1. 2. 3. 4. 5. Steady state characterization Steady state optimality Convergence Stability Scalability NE3 NE3 a2 NE2 NE1 NE1 a1 a1 a3 Optimality Convergence Scalability Stability Steady State Characterization Are As How How these do does number initial outcomes system of devices desirable? variations impact increases, impact the system the system? steady state? Is itthe possible toconditions predict behavior in the system? Do What Do these the isprocesses the outcomes steady system states will maximize impacted? lead change? to steady the system statetarget conditions? parameters? How many different outcomes are possible? How Do Is convergence previously long doesoptimal itaffected? take to steady reachstates the steady remainstate? optimal? 17/28 How Game Theory Addresses These Issues Steady-state characterization – – Steady-state optimality – in some cases Stability, scalability – – 18/28 In some special games Convergence – Nash Equilibrium existence Identification requires side information No general techniques Requires side information Nash Equilibrium Identification Time to find all NE can be significant Let tu be the time to evaluate a utility function. N Search Time: T t N A Example: u – – 19/28 i 4 player game, each player has 5 actions. NE characterization requires 4x625 = 2,500 tu Desirable to introduce side information. Example : The Cognitive Radios’ Dilemma Two cognitive radios Each radio can implement two different waveforms low-power narrowband higher power wideband Frequency domain representation of waveforms The Cognitive Radios’ Dilemma in Matrix 20/28 NE=? Repeated Games and Convergence 21/28 Repeated Game Model – Consists of a sequence of stage games which are repeated a finite or infinite number of times. – Most common stage game: normal form game. Finite Improvement Path (FIP) – From any initial starting action vector, every sequence of round robin better responses converges. Weak FIP – From any initial starting action vector, there exists a sequence of round robin better responses that converge. Better Response Dynamic 22/28 During each stage game, player(s) choose an action that increases their payoff, presuming other players’ actions are fixed. Converges if stage game has FIP. A B a 1,-1 0,2 b -1,1 2,2 Best Response Dynamic 23/28 During each stage game, player(s) choose the action that maximizes their payoff, presuming other players’ actions are fixed. converge if stage game has weak FIP. A B C a 1,-1 -1,1 0,2 b -1,1 1,-1 1,2 c 2,0 2,1 2,2 Supermodular Games Key Properties – – Why We Care – Best Response (Myopic) Dynamic Converges Nash Equilibrium Generally Exists Low level of network complexity How to Identify 2ui 0, i, j N , a A ai a j 24/28 Supermodulaar Games 25/28 NE Existence: have at least one NE. NE Identification: all NE for a game form a lattice. While this does not particularly aid in the process of initially identifying NE, from every pair of identified Convergence: have weak FIP, so a sequence of best responses will converge to a NE. Stability: if the radios make a limited number of errors or if the radios are instead playing a best response to a weighted average of observations from the recent past, play will converge. Example : outer loop power control Parameters – – – Single Cluster Pi = Pj = [0, Pmax] i,j N Utility target SINR ui pi , p j 1 abs i hi pi h j p j Nvi jN \i jN \i 26/28 Supermodular – best response convergence Summary When we use game theory to model and analyse interactive CRs, it should address : – – – – 27/28 steady state existense and identification convergence stability desirability of steady states Supermodular games : to some extent Questions? Game Theory & Cognitive Radio part B Mahdi Sadjadieh Overview 30/28 Potential Game Model Type of Potential Game Example of Exact Potential Game FIP and Potential Games How Potential Games handle the shortcomings Physical Layer Model Parameters and Potential Game Potential Game Model Existence of a potential function V such that ui bi , ai ui ai , ai V bi , ai V ai , ai ui bi , ai ui ai , ai V bi , ai V ai , ai Identification NE Properties (assuming compact spaces) – – Convergence – 31/28 NE Existence: All potential games have a NE NE Characterization: Maximizers of V are NE Better response algorithms converge. Stability – Maximizers of V are stable Design note: – If V is designed so that its maximizers are coincident with your design objective function, then NE are also optimal. Potential Games Existence of a function (called the potential function, V), that reflects the change in utility seen by a unilaterally deviating player. E1 E2 E3 E4 32/28 GPGGOPG (Gilles) OPG GPG (finite A) Potential Games 33/28 Ordinal Potential Game Identification Lack of weak improvement cycles [Voorneveld_97] FIP and no action tuples such that ui ai , ai ui bi , ai , bi ai 34/28 Better response equivalence to an exact potential game [Neel_04] Not an OPG An OPG Ordinal Potential Game Identification Lack of weak improvement cycles [Voorneveld_97] FIP and no action tuples such that ui ai , ai ui bi , ai , bi ai 35/28 Better response equivalence to an exact potential game [Neel_04] Not an OPG An OPG Other Exact Potential Game Identification Techniques Linear Combination of Exact Potential Game Forms [Fachini_97] – If <N,A,{ui}> and <N,A,{vi}> are EPG, then <N,A,{ui + vi}> is an EPG Evaluation of second order derivative [Monderer_96] 2u j 2ui , i j N , a A ai a j ai a j 36/28 Exact Potential Game Forms 37/28 Many exact potential games can be recognized by the form of the utility function Example Identification Single cluster target SINR ui p ˆ gi pi 1/ K g k pk kN \i Better Response Equivalent ui' p ˆ / K g k pk gi pi kN \i ui' p gi2 pi2 2ˆ / K gi pi Self-motivated game 2ˆ / K gi g k pi pk k N \ i V p 2ˆ / K 38/28 ˆ / K g k pk k N \ i 2 g g p p i k i k iN i k BSI game Dummy game 2 n gi2 pi2 2ˆ / K gi pi iN FIP and Potential Games 39/28 GOPG implies FIP ([Monderer_96]) FIP implies GOPG for finite games ([Milchtaich_96]) Thus we have a non-exhaustive search method for identifying when a CRN game model has FIP. Thus we can apply FIP convergence (and noise) results to finite potential games. Steady-states 40/28 As noted previously, FIP implies existence of NE Optimality If ui are designed so that maximizers of V are coincident with your design objective function, then NE are also optimal. (*) Can also introduce cost function to utilities to move NE. ui* a ui a NC a V a* ai ai 0 V In theory, can make any action tuple the NE – – 41/28 NC a* May introduce additional NE For complicated NC, might as well completely redesign ui a Convergence in Infinite Potential Games -improvement path – Approximate Finite Improvement Property (AFIP) – 42/28 Given >0, an -improvement path is a path such that for all k1, ui(ak)>ui(ak-1)+ where i is the unique deviator at step k. A normal form game, , is said to have the approximate finite improvement property if for every >0 there exists an such that the length of all -improvement paths in are less than or equal to L. [Monderer_96] shows that exact potential games have AFIP, we showed that AFIP implies a generalized -potential game. Convergence Implications 43/28 How potential games handle the shortcomings Steady-states – Optimality – – – Isolated maximizers of V have a Lyapunov function for decision rules in DV Remaining issue: – 44/28 Potential game assures us of FIP (and weak FIP) DV satisfy Zangwill’s (if closed) Noise/Stability – Can adjust exact potential games with additive cost function (that is also an exact potential game) Sometimes little better than redesigning utility functions Game convergence – Finite game NE can be found from maximizers of V. – Can we design a CRN such that it is a potential game for the convergence, stability, and steady-state identification properties AND ensure steady-states are desirable? More Examples Physical Layer Model Parameters 46/28 SINR Power Control Games Assume that there is a radio network wherein each radio can alter their power. Assume each radio reacts to some separable function of SINR, e.g. log ratio Each radio would also like to minimize power consumption Decentralized Power Control Using a dB Metric ui a fi ,1 pi , i , i fi ,2 p j j ,vi ij N i ci pi jN \i , i P a fi ,1 pi , i ,i ci pi iN 47/28 Thus game is a potential game and convergence is assured and we can quickly find steady states. Example Power Control Game Parameters – – – – Single Cluster DS-SS multiple access Pi = Pj = [0, Pmax] i,j N Utility target BER ui pi , p j 1 abs BER target Q R jN \ i W Also a potential game. 48/28 hi pi h j p j N 0 jN \ i Snapshot inner + outer loop power control Parameters – – – – Single Cluster DS-SS multiple access Pi = Pj = [0, Pmax] i,j N Utility target SINR ui pi , p j 1 abs i hi pi h j p j Nvi jN \i jN \i 49/28 Supermodular – best response convergence Game Models, Convergence, and Complexity 50/28 Determining the kind of game required to accurately model a RRM algorithm yields information about what updating processes are appropriate and thus indicates expected network complexity. In [Neel04] the following relation between power control algorithms, game models, and network complexity was observed. Summary 51/28 Distributed dynamic resource allocations have the potential to provide performance gains with reduced overhead, but introduce a potentially problematic interactive decision process. Game theory is not always applicable. Can generally be applied to distributed radio resource management schemes. Questions? Example:Exact Poential Game 53/28 return 54/28 return Example : Ordinal Poential Game 55/28 return Example : Generalized Ordinal Poential Game 56/28 return Exact Potential Game Forms 57/28 Many exact potential games can be recognized by the form of the utility function