Download Math 106 Lecture 19 Long Range Predictions with Markov Chains

Math 106
Lecture 19
Long Range Predictions with Markov Chains
1
© m j winter, 2002
Transition Matrix; Transition Diagram
Recall the cereal Jaz.
Two states: buys Jaz (state 1), and buys not-Jaz
(state 2)
To
1
From
0.2
2
1 0.8 0.2 


2 0.4 0.6 
.
from state 2
to state 1
1
0.8
2
0.4
0.6
2
1
Markov System (Chain)
• A system that can be in one of several (numbered)
states, and can pass from one state to another each
time step according to fixed probabilities.
• We use T for the transition matrix, and p for the
probability matrix (row matrix). The entries in p
represent the probabilities of finding the system in
each of the states.The sum of these probabilities is 1.
• We also regard p as describing the percentage
distribution in each state of the system.
3
probability (distribution) after n steps
If p is the initial
probability matrix, then
•pT is the probability
matrix after one step
•pT2 is the probability
matrix after 2 steps
•pTn is the probability
matrix after n steps
4
2
(Jaz) probability (distribution) after n steps
If p is the initial
probability matrix, then
In Jaz example
pT is the probability
matrix after one step
pT = [ 0.5
0.5 ]
pT2= [ 0.6
0.4 ]
pT2 is the probability
matrix after 2 steps
p = [ 0.25 0.75 ]
pT3= [ 0.64
0.36]
pT4= [ 0.656
0.344]
pTn is the probability
matrix after n steps
5
Steady State
It appears that, if we had used
pT4= [ 0.656
0.344]
as the initial probability matrix, the probability matrix after one
step would be (about) the same.
Define the equilibrium matrix, L, as the probability
matrix which is the solution to
LT = L.
Example: find the equilibrium matrix for the cereal system
6
3
Systems with non-unique solutions
Example 1
Example 2
in matrix form
3x + y = 5
AX = X
6x + 2y = 10
solution set is every
point on the line
y = 5 – 3x
clearly X = O, the
zero matrix is a
solution
7
Singular Matrix
Look at the system:
 2 1 1   x  0 
1 −2 2   y  = 0 

   
 3 −1 3   z  0 
One element of the solution
set is x = 0, y = 0, z = 0
other solutions include
x=4, y = – 3, z = – 5
x = 8, y = – 6, z = –10
If we try to find A–1 on
the calculator, we get the
message
If the matrix of coefficients is
singular, either the system
has NO solutions or it has
infinitely many
8
4
A closer look at this singular matrix
Look at the system:
with augmented matrix
 2 1 1   x  0 
1 −2 2   y  = 0 

   
 3 −1 3   z  0 
2 1 1 0
1 −2 2 0 


 3 −1 3 0 
The third row/equation is
the sum of the other two. It
is not independent.
We might be able to add another
equation and still have a solution.
9
Equilibrium Matrix for Jaz Example
Let L = [ x y ] ; know x + y = 1
Then LT=L becomes
[x
.8 .2 
y] 
 = [ x y]
.4 .6 
or
0.8x + 0.4y = x
0.2x + 0.6y = y
(0.8 − 1) x + 0.4 y = 0
0.2 x + (0.6 − 1) y = 0
−0.2 x + 0.4 y = 0
0.2 x − 0.4 y = 0
This is really only one equation.
Use the fact that x + y = 1
10
5
Solving for L
The system
L is sometimes called the (a)
steady-state probability matrix.
x+y=1
2x - 4y = 0
Technique: we used the fact that P
is a probability matrix to replace
one of the equations.
becomes
x+y=1
-6y = -2
or
x = 2/3; y = 1/3
L = [ 2/3 1/3 ] = [ .6666… .3333…. ]
11
Bookbuying Problem
Professor X is an avid reader of history books,
biographies, and mysteries. She buys and reads a
book every week.
*If she is presently reading a history book, there is a
50% chance that she will switch to a mystery the next
week.
*She never reads two biographies in a row; after a
week of reading a biography, there is a 75% chance
that she will switch to a history book the next week.
*She always reads history after reading a mystery.
*She never reads two history books in a row.
Assuming this trend continues indefinitely, what fraction
of her library will consist of biographies 1,000 weeks
from now?
12
6
Transition Diagram & Matrix
States
History
1
2
Biography
3
Mystery
.75
0
0
1
 0 .50 .50 
.75 0 .25 


0
0 
 1
2
.50
.50
1.0
.25
3
0
13
Solution - by solving LT = L
L = [x y z]
LT = L
 0 .50 .50 
[x y z] .75 0 .25  = [x y z]


0
0 
 1
Calculate solution:
use calculator
x = .47… History
y = .23… Biography
z = .29… Mystery
-1x +.75y + 1z = 0
.50x - 1y + 0z = 0
.5x +.25y -1z = 0
x + y + z =1
14
7
Long Term Transition Matrix
Assume
(1) that some power of the
transition matrix T has no zero
entries
We can use T to find the
steady-state probability
matrix.
(2) The powers of T approach a
fixed matrix T.
T is called the
steady-state transition matrix
or the
long-term transition matrix.
15
Show: For large values of n, Tn is, essentially, T
Tn *T = Tn+1 is also essentially T.
Write this equation in matrix form:
 [first row of T ] 
[second row of T ]

T =
:


 [last row of T ] 


TT=T
 [first row of T ] 
[second row of T ]


:


 [last row of T ] 


Each row of T is the steady state probability matrix L
where LT = L.
16
8
Solution- bookbuying - by taking powers of T
screen showing A^8 and
A^64
Here is A^64 stored as
matrix B
17
9