Download our presentaion - M. Javad Omidi

Game Theory & Cognitive Radio
part A
Hamid Mala
Presentation Objectives
2/28
1.
Basic concepts of game theory
2.
Modeling interactive Cognitive Radios as a
game
3.
Describe how/when game theory applies to
cognitive radio.
4.
Highlight some valuable game models.
Interactive Cognitive Radios

Adaptations of one radio
can impact adaptations of
others

Interactive Decisions

Difficult to Predict
Performance
3/28
Interactive Cognitive Radios

Scenario: Distributed SINR
maximizing power control in a
single cluster.

Final state : All nodes transmit
at maximum power.
(1) the resulting SINRs are
unfairly distributed (the closest
node will have a far superior
SINR to the furthest node)
(2) battery life would be
greatly shortened.


4/28
Power
SINR
traditional analysis techniques




5/28
Dynamical systems theory
optimization theory
contraction mappings
Markov chain theory
Research in a nutshell

6/28
Applying game theory and game models (potential and
supermodular) to the analysis of cognitive radio
interactions
–
Provides a natural method for modeling cognitive
radio interactions
–
Significantly speeds up and simplifies the analysis
process
–
Permits analysis without well defined decision
processes
Game Theory
Definition, Key Concepts
opposite color winner
Exaple
$ = card number of winner
8/28
Same color winner
opposite color winner
Exaple
$ = card number of winner
9/28
Same color winner
Exaple
Matrix representation
Girl’s strategies
Boy’s strategies
10/28
(2,-2)
(-8,8)
(-1,1)
(7,-7)
Pay-off function
Games
A game is a model (mathematical representation) of
an interactive decision situation.
Its purpose is to create a formal framework that
captures the relevant information in such a way that is
suitable for analysis.
Different situations indicate the use of different game
models.



Normal Form Game Model
1.
2.
3.
11/28
A set of 2 or more players, N
A set of actions for each player, Ai
A set of utility functions, {ui}, that describe the
players’ preferences over the outcome space
Nash Equilibrium
An action vector from which no player can profitably
unilaterally deviate.
Definition
An action tuple a is a NE if for every i  N
ui  ai , ai   ui bi , ai  for all bi Ai.
12/28
Friend or Foe Example
Friend
Friend
500,500
0,1000
Foe
1000,0
0,0
(Friend, Friend)??
13/28
Foe
No
(Friend, Foe)??
Yes
(Foe, Friend)??
Yes
(Foe, Foe)??
Yes
Modeling and Analysis Review
Modeling a Network as a Game
Network
Game
Nodes
Players
Power Levels
Actions
Algorithms
Utility Functions
Structure of game is taken
from the algorithm and the
environment
void update_power(void)
{
/*Adjusting power level*/
int k;
}
[Laboratoire de Radiocommunications
et de Traitement du Signal]
15/28
Modeling Review

The interactions in a
cognitive radio network
can be represented by
the tuple:
<N, A, {ui}, {di},T>

Timings:
–
–
–
–
16/28
Synchronous
Round-robin
Random
Asynchronous
Dynamical System
Key Issues in Analysis
1.
2.
3.
4.
5.
Steady state characterization
Steady state optimality
Convergence
Stability
Scalability
NE3
NE3
a2
NE2
NE1
NE1
a1
a1
a3
Optimality
Convergence
Scalability
Stability
Steady
State Characterization
Are
As
How
How
these
do
does
number
initial
outcomes
system
of
devices
desirable?
variations
impact
increases,
impact
the
system
the
system?
steady state?
Is
itthe
possible
toconditions
predict
behavior
in the
system?
Do
What
Do
these
the
isprocesses
the
outcomes
steady
system
states
will
maximize
impacted?
lead
change?
to steady
the
system
statetarget
conditions?
parameters?
How
many
different
outcomes
are
possible?
How
Do
Is convergence
previously
long doesoptimal
itaffected?
take to
steady
reachstates
the steady
remainstate?
optimal?
17/28
How Game Theory Addresses
These Issues

Steady-state characterization
–
–

Steady-state optimality
–

in some cases
Stability, scalability
–
–
18/28
In some special games
Convergence
–

Nash Equilibrium existence
Identification requires side information
No general techniques
Requires side information
Nash Equilibrium Identification

Time to find all NE can be significant
Let tu be the time to evaluate a utility
function.
N
Search Time: T  t  N A

Example:


u
–
–

19/28
i
4 player game, each player has 5 actions.
NE characterization requires 4x625 = 2,500 tu
Desirable to introduce side information.
Example : The Cognitive Radios’
Dilemma
Two cognitive radios
Each radio can implement two different waveforms
low-power narrowband
higher power wideband
Frequency domain
representation
of waveforms
The Cognitive Radios’
Dilemma in Matrix
20/28
NE=?
Repeated Games and Convergence

21/28
Repeated Game Model
– Consists of a sequence
of stage games which
are repeated a finite or
infinite number of times.
– Most common stage
game: normal form
game.


Finite Improvement Path
(FIP)
– From any initial starting
action vector, every
sequence of round
robin better responses
converges.
Weak FIP
– From any initial starting
action vector, there
exists a sequence of
round robin better
responses that
converge.
Better Response Dynamic


22/28
During each stage game, player(s) choose an
action that increases their payoff, presuming
other players’ actions are fixed.
Converges if stage game has FIP.
A
B
a
1,-1
0,2
b
-1,1
2,2
Best Response Dynamic


23/28
During each stage game, player(s) choose
the action that maximizes their payoff,
presuming other players’ actions are fixed.
converge if stage game has weak FIP.
A
B
C
a
1,-1
-1,1
0,2
b
-1,1
1,-1
1,2
c
2,0
2,1
2,2
Supermodular Games

Key Properties
–
–

Why We Care
–

Best Response (Myopic) Dynamic Converges
Nash Equilibrium Generally Exists
Low level of network complexity
How to Identify
 2ui
 0, i, j  N , a  A
ai a j
24/28
Supermodulaar Games
25/28

NE Existence: have at least one NE.

NE Identification: all NE for a game form a lattice.
While this does not particularly aid in the process of
initially identifying NE, from every pair of identified

Convergence: have weak FIP, so a sequence of best
responses will converge to a NE.

Stability: if the radios make a limited number of
errors or if the radios are instead playing a best
response to a weighted average of observations
from the recent past, play will converge.
Example : outer loop power control

Parameters
–
–
–
Single Cluster
Pi = Pj = [0, Pmax]  i,j N
Utility target SINR




ui  pi ,  p j   1  abs   i  hi pi   h j p j  Nvi 
jN \i
 jN \i 



26/28
Supermodular – best response convergence
Summary

When we use game theory to model and
analyse interactive CRs, it should address :
–
–
–
–

27/28
steady state existense and identification
convergence
stability
desirability of steady states
Supermodular games : to some extent
Questions?
Game Theory & Cognitive
Radio
part B
Mahdi Sadjadieh
Overview






30/28
Potential Game Model
Type of Potential Game
Example of Exact Potential Game
FIP and Potential Games
How Potential Games handle the
shortcomings
Physical Layer Model Parameters and
Potential Game
Potential Game Model
 Existence of a potential function V such that
ui  bi , ai   ui  ai , ai   V  bi , ai   V  ai , ai 
ui  bi , ai   ui  ai , ai   V bi , ai   V  ai , ai 

Identification

NE Properties (assuming
compact spaces)
–
–

Convergence
–
31/28
NE Existence: All potential
games have a NE
NE Characterization:
Maximizers of V are NE
Better response algorithms
converge.

Stability
–

Maximizers of V are stable
Design note:
–
If V is designed so that its
maximizers are coincident
with your design objective
function, then NE are also
optimal.
Potential Games

Existence of a function (called
the potential function, V), that
reflects the change in utility
seen by a unilaterally
deviating player.
E1
E2
E3
E4
32/28
GPGGOPG (Gilles)
OPG  GPG (finite A)
Potential Games
33/28
Ordinal Potential Game Identification


Lack of weak
improvement cycles
[Voorneveld_97]
FIP and no action tuples
such that
ui  ai , ai   ui  bi , ai  , bi  ai

34/28
Better response
equivalence to an exact
potential game [Neel_04]
Not an OPG
An OPG
Ordinal Potential Game Identification


Lack of weak
improvement cycles
[Voorneveld_97]
FIP and no action tuples
such that
ui  ai , ai   ui  bi , ai  , bi  ai

35/28
Better response
equivalence to an exact
potential game [Neel_04]
Not an OPG
An OPG
Other Exact Potential Game
Identification Techniques

Linear Combination of Exact Potential Game
Forms [Fachini_97]
–

If <N,A,{ui}> and <N,A,{vi}> are EPG, then
<N,A,{ui + vi}> is an EPG
Evaluation of second order derivative
[Monderer_96]
 2u j
 2ui

, i  j  N , a  A
ai a j ai a j
36/28
Exact Potential Game Forms

37/28
Many exact potential games can be
recognized by the form of the utility function
Example Identification

Single cluster target SINR
ui  p    ˆ 

gi pi


1/ K   g k pk   
 kN \i

Better Response
Equivalent




ui'  p    ˆ / K   g k pk     gi pi 
 kN \i



ui'  p    gi2 pi2  2ˆ / K gi pi

 Self-motivated game
 2ˆ / K   gi g k pi pk 

 k N \ i

V  p   2ˆ / K 
38/28



 ˆ / K   g k pk    
 k N \ i


2

g
g
p
p

i k i k 
 iN i  k

BSI game
Dummy game
2
n
    gi2 pi2  2ˆ / K gi pi 
iN
FIP and Potential Games




39/28
GOPG implies FIP ([Monderer_96])
FIP implies GOPG for finite games
([Milchtaich_96])
Thus we have a non-exhaustive search
method for identifying when a CRN game
model has FIP.
Thus we can apply FIP convergence (and
noise) results to finite potential games.
Steady-states

40/28
As noted previously, FIP implies existence of
NE
Optimality

If ui are designed so that maximizers of V are coincident with
your design objective function, then NE are also optimal.

(*) Can also introduce cost function
to utilities to move NE.
ui*  a   ui  a   NC  a 
V  a* 
ai

ai
0
V
In theory, can make any action
tuple the NE
–
–
41/28

NC  a* 
May introduce additional NE
For complicated NC, might as well
completely redesign ui
a
Convergence in Infinite Potential Games

-improvement path
–

Approximate Finite Improvement Property (AFIP)
–

42/28
Given  >0, an -improvement path is a path such that for all
k1, ui(ak)>ui(ak-1)+  where i is the unique deviator at step k.
A normal form game, , is said to have the approximate finite
improvement property if for every >0 there exists an such
that the length of all -improvement paths in  are less than
or equal to L.
[Monderer_96] shows that exact potential games
have AFIP, we showed that AFIP implies a
generalized -potential game.
Convergence Implications
43/28
How potential games handle the
shortcomings

Steady-states
–

Optimality
–
–

–
Isolated maximizers of V have a Lyapunov function for
decision rules in DV
Remaining issue:
–
44/28
Potential game assures us of FIP (and weak FIP)
DV satisfy Zangwill’s (if closed)
Noise/Stability
–

Can adjust exact potential games with additive cost function
(that is also an exact potential game)
Sometimes little better than redesigning utility functions
Game convergence
–

Finite game NE can be found from maximizers of V.
–
Can we design a CRN such that it is a potential game for
the convergence, stability, and steady-state identification
properties
AND ensure steady-states are desirable?
More Examples
Physical Layer Model Parameters
46/28
SINR Power Control Games
Assume that there is a radio network wherein each radio can alter
their power.
Assume each radio reacts to some separable function of SINR, e.g.
log ratio
Each radio would also like to minimize power consumption
Decentralized Power Control Using a dB Metric


ui  a   fi ,1 pi , i , i  fi ,2   p j j ,vi ij  N i   ci  pi 
 jN \i , 

i


P  a     fi ,1  pi , i ,i   ci  pi 


iN
47/28
Thus game is a potential game and convergence is
assured and we can quickly find steady states.
Example Power Control Game

Parameters
–
–
–
–
Single Cluster
DS-SS multiple access
Pi = Pj = [0, Pmax]  i,j N
Utility target BER






ui  pi ,  p j   1  abs  BER target  Q 

 R
jN \ i



 W


Also a potential game.
48/28


hi pi


h j p j  N 0  

jN \ i

Snapshot inner + outer loop power
control

Parameters
–
–
–
–
Single Cluster
DS-SS multiple access
Pi = Pj = [0, Pmax]  i,j N
Utility target SINR




ui  pi ,  p j   1  abs   i  hi pi   h j p j  Nvi 
jN \i
 jN \i 



49/28
Supermodular – best response convergence
Game Models, Convergence, and
Complexity


50/28
Determining the kind of game required to accurately
model a RRM algorithm yields information about what
updating processes are appropriate and thus indicates
expected network complexity.
In [Neel04] the following relation between power control
algorithms, game models, and network complexity was
observed.
Summary



51/28
Distributed dynamic resource allocations
have the potential to provide performance
gains with reduced overhead, but introduce a
potentially problematic interactive decision
process.
Game theory is not always applicable.
Can generally be applied to distributed radio
resource management schemes.
Questions?
Example:Exact Poential Game
53/28
return
54/28
return
Example : Ordinal Poential Game
55/28
return
Example : Generalized Ordinal Poential
Game
56/28
return
Exact Potential Game Forms

57/28
Many exact potential games can be
recognized by the form of the utility function