Download Markov Chains Introduction Simulation Modelling Cloud Cover Data

Markov Chains
Introduction
Simulation
Modelling Cloud Cover Data
1
Finite State Markov Chains
Definition: State Space = S = {1, 2, . . . , m}.
Definition: The sequence of random variables X1, X2, X3, . . . ,
is called a Markov chain if the Markov property holds:
P (Xn = xn|Xn−1 = xn−1, Xn−2 = xn−2, . . .) =
P (Xn = xn|Xn−1 = xn−1)
where xn, xn−1, xn−2, . . . are elements of S.
2
Example
B1, B2, . . . are independent Bernoulli random variables with parameter p. Xn =
Pn
k=1 Bk (mod
2) + 1, for n = 1, 2, . . . is a
Markov chain with state space S = {1, 2}.
P (Xn = 1|Xn−1 = 1) = 1 − p.
P (Xn = 2|Xn−1 = 1) = p.
P (Xn = 1|Xn−1 = 2) = p.
P (Xn = 2|Xn−1 = 2) = 1 − p.
3
Transition Matrix
Define a matrix P with (i, j)th entry
pij = P (Xn = j|Xn−1 = i)
pij is called the transition probability from state i to state j. P
is called a transition matrix.
All rows of P sum to one. That is,


P 
1
..
1








=
1
..
1




4
Example (cont’d)

P=

P

1
1
1-p p
p 1-p



=

1
1




5
Another Example
Set

P=




0 0.5 0.5

0 0.5 0.5  .
1 0
0
6
Example
We can study this example with R. Set up the 3 × 3 matrix P:
> P <- matrix(c(0, 0, 1, .5, .5, 0, .5, .5, 0), nrow=3)
> P
[,1] [,2] [,3]
[1,]
0
0.5
0.5
[2,]
0
0.5
0.5
[3,]
1
0.0
0.0
7
Example
Add up the rows:
> P%*%rep(1,3)
[,1]
[1,]
1
[2,]
1
[3,]
1
8
Simulating a Markov Chain
We want to simulate n values of a Markov chain having transition matrix P, starting at x1.
Function name: MC.sim
Input: n, P, x1
Output: a vector of length n
9
Reproducing Our Output
> options(width=55)
> set.seed(12867)
10
Example
Generate 20 values from the 3 × 3 transition matrix P with
starting value 3:
> MC.sim(20, P, 3)
[1] 3 1 2 2 3 1 3 1 3 1 3 1 2 3 1 2 2 3 1 3
11
Another Example

P=
0.7 0.3
0.4 0.6


> P.mat <- matrix(c(0.7,0.3,0.4,0.6),ncol=2,byrow=TRUE)
> MC.eg <- MC.sim(100, P.mat)
12
Output
> MC.eg
[1] 2 2 2 1 1 1 1 1 1 2 2 1 1 2 2 1 1 1 1 1 1 2 1 1 1
[26] 1 1 2 2 2 2 2 1 1 1 2 2 2 2 1 1 2 2 2 2 2 2 2 2 2
[51] 1 1 1 1 2 2 1 1 1 2 1 2 2 1 1 2 1 1 2 2 1 1 2 1 1
[76] 1 1 1 2 1 2 2 2 1 1 1 2 2 1 1 1 1 1 1 2 2 2 2 2 1
13
A Markov Chain Simulator
> MC.sim <- function(n,P,x1) {
+
sim <- as.numeric(n)
+
m <- ncol(P)
+
if (missing(x1)) {
+
sim[1] <- sample(1:m,1) # random start
+
} else { sim[1] <- x1 }
+
for (i in 2:n) {
+
newstate <- sample(1:m,1,prob=P[sim[i-1],])
+
sim[i] <- newstate
+
}
+ sim
+ }
14
Understanding the Code
Use of the sample() function:
> m <- 3
> sample(1:m,1,prob=c(.2, .7, .1))
[1] 2
The above simulates a random variable with distribution
p(1) = .2, p(2) = .7, p(3) = .1.
15
Understanding the Code
If the current value of the Markov Chain, having transition matrix P, is j, then the next value will be
> j <- 2
> sample(1:m,1,prob=P[j,])
[1] 2
The above simulates a random variable with distribution
P (j, 1), P (j, 2), P (j, 3). (j is 2 here.)
16
Analyzing Cloud Cover Data
> MC2 <- function(x) {
+ # Fit a 2 state MC to data in vector x (S = {1, 2})
+
n <- length(x)
+
N1 <- sum(x[-n]==1)
+
N2 <- sum(x[-n]==2)
+
N11 <- sum(x[-n]==1 & x[-1]==1)
+
N12 <- sum(x[-n]==1 & x[-1]==2)
+
N21 <- sum(x[-n]==2 & x[-1]==1)
+
N22 <- sum(x[-n]==2 & x[-1]==2)
+
P <- matrix(c(N11/N1, N21/N2, N12/N1, N22/N2), nrow=2)
+
return(P)
+ }
17
Analyzing Cloud Cover Data
> source("cloud70.R")
> # this vector contains hourly records of cloud data
> # from Winnipeg, Canada for June 1, 1970 through
> # September 30, 1970.
"1" is clear; "2" is cloudy
> # (cloudy means the measured value exceeds 2)
> cloud70[1:100]
# observed data (first 100 hours)
[1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2
[26] 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
[51] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
[76] 1 1 1 1 1 1 1 1 2 2 2 2 2 2 1 1 1 1 2 2 2 2 1 2 2
18
Analyzing Cloud Cover Data
> P <- MC2(cloud70)
> P
#
estimated transition matrix for hourly cloud cover
[,1]
[,2]
[1,] 0.8211998 0.1788002
[2,] 0.2537190 0.7462810
19
Analyzing Cloud Cover Data
Simulate data from the transition matrix in order to see if the
Markov chain model is realistic:
> cloud70.sim <- MC.sim(length(cloud70), P)
> cloud70.sim[1:100]
[1] 1 1 1 1 1 2 1 1 2 2 2 2 2 1 1 2 2 2 1 1 1 1 1 1 2
[26] 2 2 2 1 1 1 1 1 1 1 2 2 2 1 2 1 1 1 1 1 1 2 2 2 2
[51] 2 2 1 1 2 1 1 1 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 2
[76] 2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 2 2 1 1 1 1 1 1 2 2
20
Checking the Markov Chain model
Comparing runlengths in simulated and real data:
> MC2.chk <- function(x) {
+
P <- MC2(x)
+
x.sim <- MC.sim(length(x), P)
+
x.runs <- rle(x)
+
x.len <- length(x.runs[[1]])
+
x.1 <- x.runs[[1]][seq(1,x.len,2)]
+
x.2 <- x.runs[[1]][seq(2,x.len,2)]
+
x.sim.runs <- rle(x.sim)
+
x.sim.len <- length(x.sim.runs[[1]])
+
x.sim.1 <- x.sim.runs[[1]][seq(1,x.sim.len,2)]
+
x.sim.2 <- x.sim.runs[[1]][seq(2,x.sim.len,2)]
+
par(mfrow=c(1,2))
+
qqplot(x.1, x.sim.1, xlab="observed state 1 runlengths",
+
ylab="simulated state 1 runlengths")
+
abline(0,1)
+
qqplot(x.2, x.sim.2, xlab="observed state 2 runlengths",
+
ylab="simulated state 2 runlengths")
+
abline(0,1)
+ }
21
Checking the Markov Chain model
According to the following plots, the Winnipeg stays sunnier
longer than predicted by the Markov chain, but the cloudy periods are predicted well.
> MC2.chk(cloud70)
22
●
25
●
●
●
●
10
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
20
5
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
15
15
●
10
simulated state 2 runlengths
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
0
5
simulated state 1 runlengths
20
●
0
20
40
60
observed state 1 runlengths
5
10
15
20
observed state 2 runlengths