Download X - Purdue Engineering

2. Discrete Random Variables Part II: Expecta:on ECE 302 Spring 2012 Purdue University, School of ECE Prof. Ilya Pollak Expected value of X: Defini:on Ilya Pollak
Expected value of X: Defini:on E[X] is also called the mean of X
Ilya Pollak
Example: mean of a Bernoulli
random variable Ilya Pollak
Example: mean of a discrete
uniform random variable (Note: E[X] is not necessarily one of the values
that X can assume with a non-zero probability!)
Ilya Pollak
Expected value: Center of gravity of PMF •  Imagine that a box with weight pX(x) is
placed at each x.
•  Center of gravity c is the point at which
the sum of the torques is zero:
c
x
Ilya Pollak
Expected value and empirical mean • 
• 
• 
• 
Many independent Bernoulli trials with p=0.2 Bernoulli random variables X1, X2, … E[Xn]=0.2 Empirical average from many independent
experiments close to the expected value (law
of large numbers) Ilya Pollak
What E[X] is NOT •  It’s not necessarily the most likely value of X •  It’s not even always the case that P(X=E[X])>0 •  It’s not guaranteed to equal to the empirical
average Ilya Pollak
Two ways to evaluate E[g(X)] This is true because ∑ g(x)p X (x) = ∑
x
∑
y {x|g( x )= y}
= ∑y
y
g(x)
 p X (x)
y
∑
p X (x)
{x|g( x )= y}



P(g( X )= y)= pY ( y)
= ∑ ypY (y) = E[Y ]
y
Ilya Pollak
Cau:on: In general, E[g(X)]≠g(E[X]) •  Example: average speed vs average :me. •  Suppose you need to drive 60 miles. A very bad storm will hit
your area with probability 1/2. –  If the storm hits, you will drive 30 miles/hour during the en:re trip; –  If the storm does not hit, you will do 60 miles/hour during the en:re trip. •  What’s the expected value of your speed? –  (1/2) ·∙ 30 + (1/2) ·∙ 60 = 45 mph •  Is the expected value of your travel :me 60/45 = 1 hr 20 min? –  NO: T = 60/V, therefore E[T] = (1/2) ·∙ 60/30 + (1/2) ·∙ 60/60 = 1 hr 30 min –  Because you have equal chances to spend 1 hour or 2 hours driving. •  Interpreta:on: in N independent repe::ons of the trip –  You would expect to spend a total of 1.5N hours driving –  Your average speed per trip (not per unit :me traveled!) would be about
45mph Ilya Pollak
What is the meaning of E[b] if b is a
non-­‐random number? Ilya Pollak
Linearity of expecta:on Ilya Pollak
Variance and Standard Devia:on of X:
Defini:ons [
var(X ) = E ( X − E[X ])
2
]
Ilya Pollak
Calcula:ng var(X) var(X ) = E [( X − E[X ]) ] = ∑ (x − E[X ]) p (x)
2
2
X
x
= ∑ ( x 2 − 2xE[X ] + (E[X ]) 2 ) pX (x)
x
= ∑ x 2 pX (x) − 2∑ xE[X ]pX (x) + ∑ (E[X ]) 2 pX (x)
x
x
x
= ∑ x 2 p X (x) −2E[ X] ∑ xp X (x) + (E[ X])2 ∑ p X (x)
x
x
x








E[ X ]
1
E[ X 2 ]



−(E[ X ])2
= E[X 2 ] − (E[X]) 2
[
Sometimes this is easier to compute than E ( X − E[X ])
2
]
Ilya Pollak
Example 2.4: Variance of a
Bernoulli random variable Ilya Pollak
Another Bernoulli random variable Toss a coin with P(H) = p, and let
⎧ a if H
Y =⎨
⎩ b if T
⎧ p if k = a
Then pY (k) = ⎨
⎩1− p if k = b
Note :
Y − b ⎧1 if H
=⎨
=X
a − b ⎩0 if T
How to compute the expectation and
variance of this random variable?
Therefore,
Y = X(a − b) + b
E[Y] = E[X](a − b) + b = p(a − b) + b
E[Y 2 ] = E[X 2 (a − b) 2 + 2X(a − b)b + b 2 ]
= E[X 2 ](a − b) 2 + 2E[X](a − b)b + b 2
= p(a − b) 2 + 2 p(a − b)b + b 2
Ilya Pollak
Another Bernoulli random variable Toss a coin with P(H) = p, and let
⎧ a if H
Y =⎨
⎩ b if T
⎧ p if k = a
Then pY (k) = ⎨
⎩1− p if k = b
Note :
Y − b ⎧1 if H
=⎨
=X
a − b ⎩0 if T
How to compute the expectation and
variance of this random variable?
Therefore,
Y = X(a − b) + b
E[Y] = E[X](a − b) + b = p(a − b) + b
E[Y 2 ] = p(a − b) 2 + 2 p(a − b)b + b 2
var(Y ) = E[Y 2 ] − (E[Y ])2 = ( p − p 2 )(a − b)2
Ilya Pollak
Standard devia:on as measurement error •  Run a series of independent, iden:cal experiments, e.g.,
Bernoulli trials. •  Empirically es:mate the probability of an event, say, the
success in a Bernoulli trial, as the number of successes
divided by the number of experiments. •  Standard devia:on characterizes by how much we would
expect our es:mate to deviate from the actual probability of
success, over many experiments. •  For example, if p=0.2, then E[X] = 0.2, var(X)=0.16, and
standard devia:on of X is 0.4. Ilya Pollak
Root mean-­‐square devia:on of the es:mate of p
from 0.2, as a func:on of the number of trials Ilya Pollak
Standard devia:on as risk •  Suppose you have two investment opportuni:es: –  Opportunity 1: invest $1000, have equal chances of a total wipeout or
of $2000 profit –  Opportunity 2: invest $1000, earn $500 profit guaranteed. •  Expected profit for 1 is 0.5(−$1000) + 0.5($2000) = $500. •  Expected profit for 2 is $500. •  But clearly the two opportuni:es are very different: you risk a
lot under the first one, whereas the second one is riskless. •  Standard devia:on characterizes the risk: –  Opportunity 1: st.dev. = (0.5(500+1000)2 + 0.5(500−2000)2)1/2 = 1500 –  Opportunity 2: st.dev. = (1(500−500)2)1/2 = 0 •  Standard devia:on characterizes the spread of possible profits
around the expected profit. Ilya Pollak
Problem 2.20: Expecta:on and variance of
a geometric random variable As an ad campaign, a chocolate factory places
golden :ckets in some of its candy bars, with
the promise that a golden :cket is worth a trip
through the chocolate factory, and all the
chocolate you can eat for life. If the probability
of finding a gold :cket is p, find the mean and
the variance of the number of candy bars you
need to eat to find a :cket. Ilya Pollak
Mean of a geometric random variable •  Let C = # candy bars un:l 1st success •  Model C as geometric with parameter p: ⎧(1− p) k−1 p, k = 1,2,…
pC (k) = ⎨
0, otherwise
⎩
∞
E[C] = ∑ k(1− p) k−1 p
k=1
Useful fact : k(1− p)
k−1
d
= − {(1− p) k }
dp
Ilya Pollak
Mean of a geometric random variable •  Let C = # candy bars un:l 1st success •  Model C as geometric with parameter p: ⎧(1− p) k−1 p, k = 1,2,…
pC (k) = ⎨
0, otherwise
⎩
∞
E[C] = ∑ k(1− p) k−1 p
k=1
Useful fact : k(1− p)
k−1
d
= − {(1− p) k }
dp
Ilya Pollak
Mean of a geometric random variable •  Let C = # candy bars un:l 1st success •  Model C as geometric with parameter p: ⎧(1− p) k−1 p, k = 1,2,…
pC (k) = ⎨
0, otherwise
⎩
∞
E[C] = ∑ k(1− p) k−1 p
k=1
d
Useful fact : k(1− p) = − {(1− p) k }
dp
∞
∞
⎧
⎫
d
d
k
k
Therefore, E[C] = − p∑ {(1− p) } = − p ⎨∑ (1− p) ⎬
dp ⎩ k =1
⎭
k =1 dp
k−1
⎧ 1⎫ 1
d ⎧1− p ⎫
= −p ⎨
⎬ = − p⎨− 2 ⎬ =
dp ⎩ p ⎭
⎩ p ⎭ p
Ilya Pollak
How many candy bars un:l first
success? •  If there are five golden :ckets per 1,000,000
bars, then p=1/200,000. •  [Note: we assume an infinite number of bars
so that C is truly geometric.] •  Then E[C] = 200,000. •  On average, have to buy 200,000 chocolate
bars un:l first success. Ilya Pollak
E[C2] ∞
E [C 2 ] = ∑ k 2 (1− p) k−1 p
k=1
2
Another useful fact : k (1− p)
k−1
d2
d
k +1
= 2 {(1− p) } + {(1− p) k }
dp
dp
[Because the right - hand side is (k + 1)k(1− p) k−1 − k(1− p) k−1 .]
2
⎧∞
d
d ⎧∞
2
k +1 ⎫
k⎫
E ⎡⎣C ⎤⎦ =p 2 ⎨∑ (1 − p) ⎬ + p ⎨∑ (1 − p) ⎬
dp ⎩ k =1
dp ⎩ k =1
⎭

 

⎭
(1− p)2 / p
−1/ p
⎫ 1
⎫ 1 2p 1 2 1
d2 ⎧ 1
d ⎧ 1
= p 2 ⎨ − 2 + p⎬ − = p ⎨− 2 + 1⎬ − = 3 − = 2 −
dp ⎩ p
dp ⎩ p
p p
p
⎭ p
⎭ p p
Ilya Pollak
Variance of a geometric random variable 2 1 ⎛1⎞
1 1
2
2
var(C) = E [C ] − ( E[C]) = 2 − − ⎜ ⎟ = 2 −
p
p ⎝ p⎠
p
p
2
If p = 1/200,000, the standard deviation is
200,000 2 − 200,000 ≈ 200,000
Thus, your actual number candy bars until first success
may be quite far from the mean!
Ilya Pollak