Download Homework 4 Solution

Lahore University of Management Sciences
CMPE 501: Applied Probability (Fall 2010)
Homework 4: Solution
1. Random variables X and Y have joint PMF
(
c(x + y) if x ∈ {0, 1, 2}
pX,Y (x, y) =
0
otherwise
and y ∈ {1, 2}
(a) What is the value of the constant c?
Solution:
To be a valid PMF,
XX
x
pX,Y (x, y) = 1. Therefore:
y
1=
XX
x
pX,Y (x, y)
y
= c(0 + 1) + c(0 + 2) + · · · + c(2 + 2)
= 15c
This gives c = 1/15
(b) Find the marginal PMFs pX (x) and pY (y).
Solution:
pX (x) =
X
pX,Y (x, y) = c(x + 1) + c(x + 2)
y
=
2x + 3
15
for x ∈ {0, 1, 2}
Similarly
pY (y) =
X
pX,Y (x, y)
x
=
3y + 3
15
for y ∈ {1, 2}
(c) Find the conditional PMFs pX | Y (x | y) and pY
| X (y | x).
Solution:
x+y
pX,Y (x, y)
15
= 3y+3
pX | Y (x | y) =
pY (y)
15
x+y
for x ∈ {0, 1, 2} and y ∈ {1, 2}
=
3y + 3
Page 1
Similarly
pY
| X (y | x)
pX,Y (x, y)
pX (x)
x+y
for x ∈ {0, 1, 2} and y ∈ {1, 2}
=
2x + 3
=
(d) Find the conditional expectations E[X | Y = y] and E[Y | X = x].
Solution:
E[X | Y = y] =
X
x pX | Y (x | y)
x
= 0 · pX | Y (0 | y) + 1 · pX | Y (1 | y) + 2 · pX | Y (2 | y)
1+y
2+y
=0+1·
+2·
3y + 3
3y + 3
3y + 5
=
for y ∈ {1, 2}
3y + 3
Notice that E[X | Y ] does not depend on X. Similarly
X
E[Y | X = x] =
y pY | X (y | x)
y
=
3x + 5
2x + 3
for x ∈ {0, 1, 2}
which does not depend on Y .
(e) Find the expectations E[X] and E[Y ] by using
i. the marginal PMF of X (and of Y respectively), and
Solution:
E[X] =
X
x pX (x) =
19
15
X
y pY (y) =
8
5
x
Similarly
E[Y ] =
y
ii. the total expectation theorem for E[X] (and E[Y ] respectively).
Solution:
E[X] =
X
E[X | Y = y] pY (y)
y
= E[X | Y = 1] pY (1) + E[X | Y = 2] pY (2)
11 9
8 6
+
·
= ·
6 15
9 15
19
=
15
Page 2
Similarly
E[Y ] =
X
E[Y | X = x] pX (x)
x
8
=
5
(f) Are X and Y independent?
Solution:
No, X and Y are not independent. For X and Y to be independent, pX,Y (x, y) =
pX (x) pY (y) for all (x, y).
pX,Y (x, y) =
x+y
2x + 3 3y + 3
6=
·
= pX (x) pY (y)
15
15
15
In this case it would have sufficed to show that pX,Y (x, y) 6= pX (x) pY (y) for any one
value of (x, y).
2. There are two possible causes for a breakdown of a machine. To check the first possibility
would cost C1 dollars, and if that were the cause of the breakdown, the trouble could be
repaired at a cost of R1 dollars. Similarly, there are costs C2 and R2 associated with the
second possibility. Let p and 1 − p denote, respectively, the probabilities that the breakdown
is caused by the first and second possibilities. Under what conditions on p, Ci , Ri , i = 1, 2,
should we check the first possible cause of breakdown and then the second, as opposed to
reversing the checking order, so as to minimize the expected cost involved in returning the
machine to working order? Assume that if the first check is negative, you must still check the
other possibility.
Solution:
If we check the first possible cause of breakdown before the second, then:
• with probability p we will be right and the total cost would be C1 + R1 , and
• with probability 1 − p we will also have to check the second possible cause with a total
cost of C1 + C2 + R2 .
If on the other hand, we check the second possible cause of breakdown before the first, then
• with probability 1 − p we will be right and the total cost would be C2 + R2 , and
• with probability p we will also have to check the first possible cause with a total cost of
C2 + C1 + R1 .
Let T1 be the total cost if we check the first cause before checking the second cause. Similarly
let T2 be the total cost if we check the second cause before the first cause. Because of the
above reasoning, its clear that T1 and T2 are r.v.s with the following PMFs.
(
p
pT1 (k) =
1−p
(
p
pT2 (k) =
1−p
if k = C1 + R1
if k = C1 + C2 + R2
Page 3
if k = C2 + C1 + R1
if k = C2 + R2
Now, we should check the first possible cause of breakdown and then the second iff E[T1 ] < E[T2 ].
This means that:
E[T1 ] < E[T2 ]
which expands to
p(C1 + R1 ) + (1 − p)(C1 + C2 + R2 ) < p(C2 + C1 + R1 ) + (1 − p)(C2 + R2 )
After some cancellations of common terms, this simplifies to
p
C1
>
1−p
C2
or
p>
C1
C1 + C2
So we should check the first possible cause before the second possible cause iff p >
C1
C1 + C2
3. Professor May B. Right often has her science facts wrong, and answers each of her students’
questions incorrectly with probability 1/4, independently of other questions. In each lecture
Professor Right is asked either 1 or 2 questions with equal probability. Assume that Professor
Right has 20 lectures each semester and each lecture is independent of any other lecture.
(a) What is the probability that Professor Right give wrong answers to all the questions she
gets in a given lecture?
Solution:
Let X be the number of questions asked A be the event that Professor Right gives wrong
answer to all the questions asked. Using the total probability theorem:
X
P(A) =
P(X = x) P(A | X = x)
x
= P(X = 1) P(A | X = 1) + P(X = 2) P(A | X = 2)
1 1 1 1
= · + ·
2 4 2 16
5
=
32
(b) Given that Professor Right gave wrong answers to all the questions she was asked in a
given lecture, what is the probability that she got two questions?
Solution:
Let the r.v. X and event A be defined as in part (a) above. We are required to find
Page 4
P(X = 2 | A).
P({X = 2} ∩ A)
P(A)
P(X = 2) P(A | X = 2)
=
5/32
1
1
·
= 2 5 16
P(X = 2 | A) =
32
1
=
5
(c) Let X and Y be the number of questions asked and the number of questions answered
correctly in a lecture, respectively. What is the mean and variance of X and the mean
and variance of Y ?
Solution:
The PMF pX (x) of X is given by:
(
1/2 if x ∈ {1, 2}
pX (x) =
0
otherwise
2
With this PMF, the E[X] and E X can be calculated as 1.5 and 2.5 respectively. This
gives
X
3
E[X] =
x pX (x) =
2
x
and
1
var(X) = E X 2 − E[X]2 =
4
For Y , it’s easy to see that the conditional PMF pY | X (y | x) is a binomial r.v. with n = x
and p = 3/4. So
y x−y
3
x
1
for 0 ≤ y ≤ x ≤ 2
pY | X (y | x) =
y
4
4
Now using the total probability theorem, the (marginal) PMF of Y can be calculated
as:


pX,Y (1, 0) + pX,Y (2, 0) if y = 0


p
X,Y (1, 1) + pX,Y (2, 1) if y = 1
pY (y) =

pX,Y (2, 2)
if y = 2



0
otherwise
Using pX,Y (x, y) = pX (x) pY
| X (y | x)


5/32



9/16
pY (y) =

9/32



0
we get
if y = 0
if y = 1
if y = 2
otherwise
Page 5
With this PMF, the E[Y ] and E Y 2 can be calculated as 9/8 and 27/16 respectively.
This gives
X
9
y pY (y) =
E[Y ] =
8
y
and
27
var(Y ) = E Y 2 − E[Y ]2 =
64
(d) Give a neatly labeled table of the joint PMF pX,Y (x, y).
Solution:
y
pX,Y (x, y)
2
0
9/32
1
3/8
3/16
0
1/8
1/32
1
2
x
(e) Let Z = X + 2Y . What are the expectation and variance of Z?
Solution:
From the joint PMF pX,Y (x, y) above we can calculate the PMF of Z directly as:


1/8 if z = 1




1/32 if z = 2



3/8
if z = 3
pZ (z) =

3/16 if z = 4





9/32 if z = 6



0
otherwise
2
Using this we can easily calculate E[Z] and E Z as 15/4 and 67/4 respectively. This
gives
X
15
E[Z] =
z pZ (z) =
4
z
and
43
var(Z) = E Z 2 − E[Z]2 =
16
We could also have found E[Z] using
15
4
as before. Notice that this does NOT require X and Y to be independent. We could
not, however have used var(Z) = var(X) + 4var(Y ) since X and Y are dependent and
we had to resort to finding the PMF of Z first.
E[Z] = E[X + 2Y ] = E[X] + 2E[Y ] =
Page 6
(f) The university where Professor Right works has a peculiar compensation plan. For each
lecture, she gets paid a base salary of $1,000 plus $40 for each question she answers
and an additional $80 for each of these she answers correctly. In terms of the random
variable Z, she gets paid $1000 + $40Z per lecture. What are the expected value and
variance of her semesterly salary?
Solution:
Let Li (for 1 ≤ i ≤ 20) be the r.v. representing Professor Right’s salary for the ith lecture.
Also let Zi (for 1 ≤ i ≤ 20) be the r.v. so that Li = 1000 + 40Zi . Her semesterly salary
S is then given by:
S=
20
X
Li =
20
X
(1000 + 40Zi ) = 20000 + 40
Zi
i=1
i=1
i=1
20
X
By linearity of expectations:
E[S] = 20000 + 40E
"
20
X
#
Zi = 2000 + 40 · 20E[Zi ] = 23000
i=1
Since all Li are independent:
!
20
20
X
X
var(Li ) = 20 var(1000 + 40Zi ) = 20 · 402 var(Zi ) = 36000
Li =
var(S) = var
i=1
i=1
(g) Determined to improve her reputation, Professor Right decides to teach an additional
20-lecture course in her specialty (math), where she answers questions incorrectly with
probability 1/10 rather than 1/4. What is the expected number of questions that she
will answer wrong in a randomly chosen lecture (math or science)?
Solution:
Let Am (and As ) be the event that the randomly chosen lecture is math (and science
respectively). Also, let Y be the number of questions she answers incorrectly. We are
required to find E[Y ]. We first find the conditional PMFs pY | Am (y) and pY | As (y) using
the method similar to the one used in part (c).




21/32
if
y
=
0
171/200 if y = 0






5/16

if y = 1
7/50
if y = 1
pY | Am =
pY | As =


1/32
if y = 2
1/200
if y = 2






0

otherwise
0
otherwise
Using these we can calculate the values for E[Y | As ] and E[Y | Am ] as:
X
E[Y | As ] =
y pY | As (y) = 3/8
y
and
E[Y | Am ] =
X
y
Page 7
y pY
| Am (y)
= 3/20
E[Y ] can now be calculated using the total expectation theorem.
E[Y ] = P(Am ) E[Y | Am ] + P(As ) E[Y | As ]
1 3
1 3
= ·
+ ·
2 20 2 8
21
=
80
4. Joe Lucky plays the lottery on any given week with probability p, independently of whether
he played on any other week. Each time he plays, he has a probability q of winning, again
independently of everything else. During a fixed period of n weeks, let X be the number of
weeks that he played the lottery and Y the number of weeks that he won.
(a) What is the probability that he played the lottery any particular week, given that he
did not win anything that week?
Solution:
Let Li be the event that he played the lottery in week i and Wi be the event that he
won something in week i. We are required to find P(Li | Wic ).
P(Li | Wic ) =
=
=
=
=
P(Li ∩ Wic )
P(Wic )
P(Li ∩ Wic )
P(Li ∩ Wic ) + P(Lci ∩ Wic )
P(Li ) P(Wic | Li )
P(Li ) P(Wic | Li ) + P(Lci ) P(Wic | Lci )
p · (1 − q)
p · (1 − q) + (1 − p) · 1
p − pq
1 − pq
You can save the first two steps if you remember Baye’s rule.
(b) Find the conditional PMF pY
| X (y | x).
Solution:
Conditioned on X, Y has a binomial PMF:
x y
pY | X (y | x) =
q (1 − q)x−y
y
for 0 ≤ y ≤ x
(c) Find the joint PMF pX,Y (x, y).
Solution:
X has a binomial PMF with pX (x) =
given by:
$n
x
px (1 − p)n−x . The joint PMF pX,Y (x, y) is then
pX,Y (x, y) = pX (x) pY | X (y | x)
n x
n−x x
q y (1 − q)x−y
p (1 − p)
=
y
x
Page 8
for 0 ≤ y ≤ x ≤ n
which simplifies to
n!
px q y (1 − p)n−x (1 − q)x−y
(n − x)! (x − y)! y!
pX,Y (x, y) =
for 0 ≤ y ≤ x ≤ n
(d) Find the marginal PMF pY (y). (Hint: One possibility is to start with the answer to
part (c), but the algebra can be messy. But if you think intuitively about the procedure
that generates Y , you may be able to guess the answer.)
Solution:
The probability that in any week Joe will win something is P(Li ) P(Wi | Li ) = pq. So Y
is a binomial r.v. with parameters n and pq.
n
(pq)y (1 − pq)n−y for 0 ≤ y ≤ n
pY (y) =
y
(e) Find the conditional PMF pX | Y (x | y). Do this algebraically using previous answers.
Solution:
pX | Y (x | y) =
=
=
pX,Y (x, y)
pY (y)
n!
x y
n−x (1 − q)x−y
(n−x)! (x−y)! y! p q (1 − p)
n!
y
n−y
y!(n−y)! (pq) (1 − pq)
px−y (1 − q)x−y (1 − p)n−x
(n − y)!
(n − x)!(x − y)!
·
(1 − pq)n−y
which simplifies to
pX | Y (x | y) =
n−y
x−y
1−p
1 − pq
n−x p − pq
1 − pq
x−y
for 0 ≤ y ≤ x ≤ n
(f) (Optional) Rederive the answer to part (e) by thinking as follows: For each one of the
n − Y weeks that he did not win, the answer to part (a) should tell you something.
Solution:
Given that Y = y, Joe did not win anything in n − y weeks. For each of these weeks,
the probability that he played the lottery (given that he did not win anything) is given
by P(Li | Wic ) = p−pq
1−pq as calculated in part (a). So, given Y = y the r.v. X taking the
value x means that Joe did not play the lottery in x − y of the n − y weeks he did not
win anything. So now given Y = y, X is a binomial r.v. with parameters n − y and
p−pq
1−pq . Therefore
pX | Y (x | y) =
n−y
x−y
p − pq
1 − pq
x−y p − pq
1−
1 − pq
(n−y)−(x−y)
for 0 ≤ y ≤ x ≤ n
which is the identical to the conditional PMF pX | Y (x | y) calculated in part (e).
5. Let X and Y be random variables that take on values from the set {-1, 0, 1}.
Page 9
(a) Find a joint PMF assignment for which X and Y are independent, and confirm that X 2
and Y 2 are then also independent.
Solution:
Let Xsq and Ysq be the r.v.s corresponding to the X 2 and Y 2 respectively. For the PMFs
given below, it can be verified that if X and Y are independent then X 2 and Y 2 are also
independent.
pX,Y (x, y)
y
ysq
pXsq ,Ysq (xsq , ysq )
1
1/9
1/9
1/9
0
1/9
1/9
1/9
1
2/9
4/9
−1
1/9
1/9
1/9
0
1/9
2/9
−1
0
1
0
1
x
xsq
(b) Find a joint PMF assignment for which X and Y are not independent, but for which
X 2 and Y 2 are independent.
Solution:
For the PMFs given below, it can be verified that X 2 and Y 2 are independent but X
and Y are not.
ysq
pXsq ,Ysq (xsq , ysq )
pX,Y (x, y)
y
1
0
1/9
2/9
1
2/9
4/9
0
1/9
1/9
1/9
0
1/9
2/9
−1
2/9
1/9
0
0
1
−1
0
1
xsq
x
6. Pascal PMF. You toss a coin until exactly k heads appear. Assume that the probability of
a head on any toss is p independent of all other tosses.
(a) Find the probability that the k = 5th head appears on the y = 10th toss?
Solution:
The k = 5th head on the y = 10th toss means that we have exactly k − 1 = 4 heads
on the first y − 1 = 9 tosses and a$ head
on the 10th toss. The probability of exactly 4
heads on first 9 heads is given by 94 p4 (1 − p)5 and the probability of the (independent)
10th toss resulting in a head is p. The required probability is therefore given by:
9 5
9 4
5
p (1 − p)5
p (1 − p) p =
P(5th head on 10th toss) =
4
4
(b) Let Y be the toss number that results in the kth head. Find the PMF of Y .
Solution:
Page 10
By similar reasoning, the PMF of Y is given by:
y−1 k
p (1 − p)y−k
pY (y) =
k−1
for y ≥ k
This is known as the Pascal PMF of order k.
7. The Cauchy-Schwarz inequality tells us that for two vectors u and v in an inner product
space,
|hu, vi| ≤ kuk · kvk
with the equality holding iff one vector is a constant multiplier of the other. Prove the
analogue of the Cauchy-Schwartz inequality for random variables:
p
p
E[XY ] ≤ E[X 2 ] E[Y 2 ]
(Hint: Use the fact that E (αX + Y )2 ≥ 0 for all real (constants) α).
Solution:
Starting from the hint:
E (αX + Y )2 ≥ 0
for all real (constants) α. Expanding the L.H.S.
E α2 X 2 + 2αXY + Y 2 ≥ 0
By linearity of expectations, this becomes
α2 E X 2 + 2αE[XY ] + E Y 2 ≥ 0
This is a quadratic equation in α which is ≥ 0 for all real α. This means that the discriminant ≤ 0. Therefore:
$ $ 2 (2 E[XY ])2 − 4 E X 2
E Y
≤0
from which the desired result follows directly.
8. Show the following properties for expectations:
(a) Pull-through property:
E[Y g(X) | X] = g(X)E[Y | X] for any suitable function g
Solution:
E[Y g(X) | X] =
X
y
Page 11
yg(X)pY
| X (y | X)
Since the summation is over y, g(X) can be pulled out
E[Y g(X) | X] = g(X)
X
ypY
| X (y | X)
y
The summation on the R.H.S. is just the E[Y | X] and so
E[Y g(X) | X] = g(X)E[Y | X]
(b) Tower property:
E[E[Y | X, Z] | X] = E[Y | X]
Solution:
We start by calculating the inner expected value on the L.H.S.
X
E[Y | X, Z] =
y pY | X,Z (y | X, Z)
y
(1)
X P(Y = y, X = x, Z = z)
=
y
P(X = x, Z = z)
y
Notice that this is a function of X and Z. Let h(X, Z) = E[Y | X, Z]. So now:
E[E[Y | X, Z] | X] = E[h(X, Z) | X]
X
h(X, z)pZ | X (z | X)
=
(2)
z
=
X
z
P(X = x, Z = z)
h(X, z)
P(X = x)
Replacing h(X, Z) = E[Y | X, Z] from equation (1) into equation (2) gives:
E[E[Y | X, Z] | X] =
=
X X P(Y = y, X = x, Z = z) P(X = x, Z = z)
y
P(X = x, Z = z)
P(X = x)
z
y
X X P(Y = y, X = x, Z = z)
y
P(X = x)
z
y
The summation over y can be pulled out to get
E[E[Y | X, Z] | X] =
X X pY,X,Z (y, x, z)
y
pX (x)
z
y
P
P
Just as x pY,X (y, x) = pY (y), the summation z pY,X,Z (y, x, z) = pY,X (y, x). This
means that the above now simplifies to:
E[E[Y | X, Z] | X] =
X pY,X (y, x) X
y
=
y pY
pX (x)
y
y
which completes the proof.
Page 12
| X (y | x)
= E[Y | X]
(c) E[E[Y | X] | X, Z] = E[Y | X]
Solution:
Since E[Y | X] is a function of X, we let g(X) = E[Y | X]. Now:
E[E[Y | X] | X, Z] = E[g(X) | X, Z]
From part (a) we have E[Y g(X) | X] = g(X)E[Y | X] and so this simplifies to:
E[E[Y | X] | X, Z] = g(X)E[1 | X, Z]
= g(X) · 1
= E[Y | X]
Optional Questions
Here we provide only the final solution to these optional questions. In some cases, hints are also
provided. In case you are unable to calculate the given answer, please contact the TAs or the
instructor.
1. (Chapter 2 Problem 18): PMF of the minimum of several random variables. On
any given day your golf score is an integer between 101 and 110, each with probability 0.1.
Determined to improve your score, you decide to play on three different days and declare as
your score the minimum X of the scores X1 , X2 and X3 on the three days.
(a) Calculate the PMF of X.
Solution:
See text book.
(b) By how much has your expected score improved as a result of playing on three days?
Solution:
See text book.
2. A sample of 3 items is selected at random from a box containing 20 items of which 4 are
defective. Find the expected number of defective items in the sample.
Solution:
4
16
k
3−k
pX (k) =
20
3
for 0 ≤ k ≤ 3
where X is the number of defective items in the sample. Using this we have:
E[X] = 0.6
Page 13
3. A newsboy purchases papers at 10 cents and sells them at 15 cents. However, he is not allowed
to return unsold papers. If his daily demand is a binomial random variable with n = 10 and
p = 1/3, approximately how many papers should he purchase so as to maximize his expected
profit?
Solution:
Let X be the demand and Y be the profit. X is a binomial r.v. with n = 10 and p = 1/3.
(
15X − 10a if X < a
pY (y) =
5a
if X ≥ a
where a is the number of papers he bought. Using the total expectation theorem, we can find
the value of a which maximizes E[Y ] (this might be done graphically). The value of a that
maximizes his expected profit is a = 3.
4. A certain airline company, having observed that five percent of the people making reservation
on a flight do not show up for the flight, sells hundred tickets for a plane that has only ninety
five seats.
(a) What is the exact probability that there will be a seat available for every person who
shows up for the flight?
Solution:
P(seat for every person)=0.564
(b) Recalculate the above probability using a Poisson approximation.
Solution:
Using the Poisson approximation we get P(seat for every person)=0.56
5. Suppose two million lottery tickets are issued with 100 winning tickets among them.
(a) If a person purchases 100 tickets, what is the probability he will win the lottery?
Solution:
The probability of winning ≈ 0.005.
(b) How many tickets should you buy so that the probability that you have a winning ticket
is greater than 95%?
Solution:
You need to buy approximately 60,000 tickets to be more than 95% confident of having
a winning ticket.
Page 14