Download university of oslo

UNIVERSITY OF OSLO
Faculty of Mathematics and Natural Sciences
Examination in
STK2130 — Modeling by stochastic processes.
Day of examination: Friday, June 10, 2011.
Examination hours:
14.30 – 18.30.
This problem set consists of 4 pages.
Appendices:
Formulary
Permitted aids:
Approved calculator.
Please make sure that your copy of the problem set is
complete before you attempt to answer anything.
Problem 1. (10 points)
Consider a Markov chain Xn , n ≥ 0 with state

3
0 20
 0 1
 1 1
P =
 2 2
 0 0
0 0
space I = {1, 2, 3, 4, 5} and transition matrix
1
1
3 
2
5
20
0
0
0
0
0
0
0
0
1
3
1
2
2
3
1
2


.


(i) Describe the Markov chain by a diagram.
(ii) Find all communicating classes of the Markov chain. Which are closed ?
(iii) Calculate for all i, j = 1, ..., 5 the probability that X2 = j given X0 = i.
(iv) Which classes are recurrent and which classes are transient ? Find the period of the state
i = 1.
Problem 2. (10 points)
Let us consider a stock price process Sn with values
Sn = 0 $ eller 1 $ eller 2 $
for all n = 0, 1, 2, ...(in weeks).
Assume that (Sn )n≥0 is a Markov chain with transition matrix


1
0
0
P =  0.2 0.48 0.32  .
0.1 0.6 0.3
(Continued on page 2.)
Examination in STK2130, Friday, June 10, 2011.
Page 2
(i) Find for all i = 0, 1, 2 the probability that the stock price starting in i $ ever reaches 0 $.
(ii) Compute for all i = 0, 1, 2 the mean time for the stock price to hit 0 $ given that its initial
value is i $.
(iii) A trader on a market, who observes the stock prices Sn , n = 0, 1, 2, ..., wants to buy the
stock, if it costs 1 $.
What is the expected value of the stock price three weeks after the purchase, that is what is
E[ST +3 |T < ∞] ?
Here T = H A , that is the hitting time
H A = inf{n ≥ 0 : Sn ∈ A},
where A = {1}.
(iv) What is the mean time after the first week that the stock price ever reaches the values 0
$ or 2 $, given that S0 = 2 $, that is what is
E T A |S0 = 2] ?
Here T A is defined as
T A = inf{n ≥ 1 : Sn ∈ A},
where A = {0, 2}.
Problem 3. (Restocking of a warehouse) (10 points)
Let us consider a warehouse with a capacity of c units of a stock. Denote by Dn the demand
for the stock in time period n. Further, let Xn be the residual stock at the end of time period
n. The warehouse manager restocks to capacity c in the beginning of time period n + 1, if
Xn ≤ m,
where m ∈ {0, 1, ..., c − 1} is a certain threshold. So Xn , n ≥ 0 is given by
(c − Dn+1 )+
if Xn ≤ m
Xn+1 =
,
(Xn − Dn+1 )+ if m < Xn ≤ c
def
where (a)+ = max(a, 0) for a ∈ R.
Suppose that Dn , n ≥ 1 are independent and identically (i.i.d) distributed random variables
with common distribution D such that
P (D ≥ i) = 4−i ,
that is P (D = i) = 4−i − 4−(i+1) , i ≥ 1 (D ∈ {0, 1, 2, ...}). In addition, assume that c = 2 (i.e.
m ∈ {0, 1}).
(Continued on page 3.)
Examination in STK2130, Friday, June 10, 2011.
Page 3
Find the optimal restocking strategy of the warehouse manager, that is minimize the cost
function
ag(m) + bf (m)
with respect to m, where g(m) stands for the long-run frequency of restocking and where f (m)
represents the long-run proportion of unmet demand. Further a ≥ 0 is the cost for restocking
and b ≥ 0 is the (lost) profit with respect to unmet demand per unit stock. Assume that a =
2 $.
Hint: Formulary.
Problem 4. (10 points)
Consider a Markov chain Xn , n ≥ 0 on the state space I = {0, 1, 2, ...} with transition
probabilities given by
p01 = 1, pi,i+1 + pi,i−1 = 1, pi,i+1 =
i+1
i
2
pi,i−1 , i ≥ 1.
Find the probability that Xn ≥ 1 for all n ≥ 1, given that X0 = 0, , that is
P (Xn ≥ 1 for all n ≥ 1 |X0 = 0) .
Hint: Use substitution of the form ui = hi−1 − hi and the telescope sum. See the formulary.
End
(Continued on page 4.)
Examination in STK2130, Friday, June 10, 2011.
Page 4
Appendix: Formulary
a) hA
i , i ∈ I (hitting probability) is the minimal non-negative solution to
A =1
hP
if i ∈ A
i
.
A
A
hi = j∈I pij hj if i ∈
/A
b) kiA , i ∈ I (mean hitting time) is the minimal non-negative solution to
kiAP
=0
if i ∈ A
.
A
A if i ∈
ki = 1 + j ∈A
p
k
/A
ij
j
/
c) (i)
n−1
c
j=0
i=0
X
1X
f (m) = lim
f (Xj ) =
πi f (i),
n−→∞ n
def
where π = (π0 , ..., πc ) is the invariant distribution and
h
E [(D − c)+ ]
if i ≤ m
def
e
f (i) = E Xn+1 |Xn = i] =
+
E [(D − i) ] if m < i ≤ c
(expected unmet demand)
(ii) Y ∈ {0, 1, 2, ...} random variable =⇒
E[Y ] =
X
P (Y ≥ k).
k≥1
(iii)
X
k≥0
qk =
1
, 0≤q<1
1−q
(iv)
n−1
c
j=0
i=0
X
1X
g(m) = lim
g(Xj ) =
πi g(i),
n−→∞ n
def
where π = (π0 , ..., πc ) is the invariant distribution and
1 if 0 ≤ i ≤ m
def
g(i) =
.
0
else
(v)
pij =
P ((c − D)+ = j) if 0 ≤ i ≤ m
.
P ((i − D)+ = j) if m < i ≤ c
d) (i)
X 1
π2
=
.
n2
6
n≥1
(ii) Telescope sum:
m
X
i=1
(hi−1 − hi ) = h0 − hm .