Download Discrete Random Variables: Expectation, Mean and Variance

Discrete Random Variables: Expectation,
Mean and Variance
Berlin Chen
Department of Computer Science & Information Engineering
National Taiwan Normal University
Reference:
- D. P. Bertsekas, J. N. Tsitsiklis, Introduction to Probability , Sections 2.3-2.4
Motivation (1/2)
• An Illustrative Example: Suppose that you spin the
wheel k times, and that k i is the number of times that
the outcome (money) is mi (there are n distinct
outcomes, m1 , m 2 ,  , m n )
• What is the amount of money that you “expect” to get
“per spin”?
– The total amount received is
m1k1  m 2 k 2    m n k n
– The amount received per spin is
m1k1  m 2 k 2    m n k n
M 
k
Probability-Berlin Chen 2
Motivation (2/2)
– If the number of spins k is very large, and if we are willing to
interpret probabilities as relative frequencies, it is reasonable to
anticipate that mi comes up a fraction of times that is roughly
equal to p i
ki
pi 
k
– Therefore, the amount received per spin can be also
represented as
m1k1  m 2 k 2    m n k n
M 
k
 m1 p1  m 2 p 2    m n p n
Probability-Berlin Chen 3
Expectation
• The expected value (also called the expectation or the
mean) of a random variable X , with PMF p X , is
defined by
E X    xp X x 
x
– Can be interpreted as the center of gravity of the PMF
(Or a weighted average, in proportion to probabilities, of the
possible values of X )
• The expectation is well-defined if
 x p X x   
x
– That is,
 xp X x  converges to a finite value
x
 x  c  p X x   0
x
 c   x  p X x 
x
Probability-Berlin Chen 4
Moments
• The n-th moment of a random variable X is the
expected value of a random variable X n (or the random
variable Y , Y  g  X   X n )
   x
E X
n
x
n
p X x 
– The 1st moment of a random variable
expectation)
Xn
X
is just its mean (or
is termed as X raised to the power of n (or the nth power),
or the nth power of X.
Probability-Berlin Chen 5
Expectations for Functions of Random Variables
• Let X be a random variable with PMF p X , and let g  X 
be a function of X . Then, the expected value of the
random variable g  X  is given by
x1
E g  X
 
x3
x2
x4
x5
x6
x7
 g x  p X x 
x
y2
y1
y3
y4
• To verify the above rule
 p X x 
– Let Y  g  X  , and therefore pY  y  
x g  x  y 
E g  X
y
y
  E Y    y p Y  y 
y

x g  x  y 
p X x   
  g x  p X x 
x
y
?

g x  p X x 
x g  x  y 
Probability-Berlin Chen 6
Variance
• The variance of a random variable X is the expected
value of a random variable  X  E  X 2
var  X



2 
2
 x  E X  p X x 
E  X  E X
x
– The variance is always nonnegative (why?)
– The variance provides a measure of dispersion of X around its
mean
– The standard derivation is another measure of dispersion, which
is defined as (a square root of variance)

X

var  X

• Easier to interpret, because it has the same units as
X
Probability-Berlin Chen 7
An Example
• Example 2.3: For the random variable X with PMF
1 / 9, if x is an integer in the range [-4, 4],
p X x   
otherwise
 0,
Discrete Uniform Random Variable
1 4
E X    x p X x  
x0
9 x  4
x


var X   E  X  EX    x  EX 
2
2
x
Or, let Z   X  EX   X 2
2
2/9, if z  1,4,9,16

 pZ z   1 / 9, if z  0
0, otherwise

60
var  X   EZ    z pZ z  
9
z
1 4 2 60
p X x  
x 
9 x  4
9
Probability-Berlin Chen 8
Properties of Mean and Variance (1/2)
• Let X be a random variable and let
Y  aX  b
where a and
Then,
a linear function of X
b are given scalars
E Y   a E X   b
var Y   a 2 var  X 
• If g  X

is a linear function of
E g  X   g E X 
X , then
How to verify it ?
Probability-Berlin Chen 9
Properties of Mean and Variance (2/2)


 
EY    ax  b pX x   a xpX x   b pX x   aEX   b
x

 x
  x
1
varY    ax  b  EaX  b2 pX x 
x
  ax  b  aEX   b2 pX x 
x
 a  x  EX  pX x 
2
2
x
 a var X 
2
var X 
Probability-Berlin Chen 10
Variance in Terms of Moments Expression
• We can also express variance of a random variable X
as
 
var  X   E X 2  E X 2
var X   x  EX  pX x
2
x


  x2  2xEX   EX  pX x
2
x





2
2
  x pX x  2EX  xpX x  EX   pX x
x

x

x

 
 EX  EX 
 E X 2  2EX   EX 
2
2
2
2
Probability-Berlin Chen 11
An Example
• Example 2.4: Average Speed Versus Average Time. If
the weather is good (with probability 0.6), Alice walks the 2 miles to
class at a speed of V=5 miles per hour, and otherwise rides her
motorcycle at a speech of V=30 miles per hour. What is the
expected time E[T] to get to the class ?
 0 .6 ,
pV v   
 0 .4 ,
2
T  g V  
V

 0 .6,
 pT t   
 0 .4,

if v  5
if v  30
E[V ]  0 .6  5  0 .4  30  15
E[T ]  0 .6 
if t 
2
5
2
if t 
30
2
2
4
 0 .4 

5
30 15
However, E[T ]  E[ g V ]  g E[V ] 
2
15
Probability-Berlin Chen 12
Mean and Variance of the Bernoulli
• Example 2.5. Consider the experiment of tossing a
biased coin, which comes up a head with probability p
and a tail with probability 1  p , and the Bernoulli
random variable with PMF
 p,
p X x   
1  p ,
if x  1
if x  0
E X    xp X x   1  p  0  1  p   p
 
x
E X 2   x 2 p X x   12  p  0 2  1  p   p
x
 
var  X   E X 2  E X 2  p  p 2  p 1  p 
Probability-Berlin Chen 13
Mean and Variance of the Discrete Uniform
• Consider a discrete uniform random variable with a
nonzero PMF in the range [a, b]
1

,

p X x    b  a  1
0,

if x  a , a  1,  , b
otherwise
b
ab
1

x
E X    xp X  x  



b
a
1
2
x
x a
 
EX
2
b
1
1
 bb  12b  1 a  1a 2a  1 
2




x



b  a  1 x a
b  a 1 
6
6

1
 bb  12b  1 a  1a 2a  1   a  b 
2


var X   E X 2  EX  


b  a 1 
6
6
2
 

b  a b  a  1b  a  2  b  a b  a  2
1


b  a 1
12
12
 
2
Probability-Berlin Chen 14
Mean and Variance of the Poisson
• Consider a Poisson random variable with a PMF
p X x   e  
x



EX    xpX x   xe
x!
x 0
x
x
x!
x  0 ,1,2,  ,
,

   e 
x 1
x 1
x  1!

  e
x


x!
x  0

1
 

E X   x pX x    x e
2
2
2 
x 0
x

   x  1e
x  0
 

x
x!

  x
x 1
x 1
x  1!
x   
x 
 






   EX   1  2  
  e
    xe
x!
x!   x0
x! 
 x0
x
var X   E X 2  EX 2  2    2  
1
Probability-Berlin Chen 15
Mean and Variance of the Binomial
• Consider a binomial random variable with a PMF
n x
p X x     p 1  p n  x , x  0 ,1, ,n
 x
n
n
n
n x
n x
n x
n x
EX    xpX x    x  p 1  p    x  p 1  p    x
x 0
x
 x
x 1
 x
x 1
n!
n x
p x 1  p 
x!n  x !
n 1

n 1! p x 1  p n1 x  np
n  1!
n x
x 1
p 1  p   np
 np
x 1  x  1!n  x !
x0 x!n  1  x!
n
  

 n
EX  X   EX  X  1   xx  1  p 1  p 
x
E X  E X  X  EX  (to be verified later on!)
2
2
n
2
x
 
x 0
 nn  1 p
 
2
n x
1
n
  xx  1
x 2
n!
n x
p x 1  p 
x!n  x !
n  2! p x2 1  pn x  nn 1 p 2

x  2  x  2!n  x !
n

1

var X   E X  EX   E X 2  X  EX   EX 
2
2
2
 nn  1 p 2  np  n 2 p 2  np1  p 
Probability-Berlin Chen 16
Mean and Variance of the Geometric
• Consider a geometric random variable with a PMF
p X x   1  p  p , x  1,2 ,  ,
E X    xp  x    x 1  p  p  p  xq
(let q  1-p  1)
x 1

X

x 1
x 1
x 0
x
x 1
 1 
  x

d 
d q 
1

q
1
 p 1

 p  x 1   p 
dq
dq
1  q 2 p
  

EX  X   E X  X  1   x  x  11  p 
E X 2  E X 2  X  E X  (to be verified later on! )

2
x 1
x 0

 pq  x  x  1q x  2
 
x2


p  pq  x  x  1q x  2
(let q  1-p  1)
x2
 1 

d 2 
1 q
2
21-p 

 pq  2   pq
p2
d q
1  q 3

var  X   E X 2  E X   E X 2  X  E X   EX 
1  p 
21-p  1
1




p2
p p2
p2
2
2
Probability-Berlin Chen 17
An Example
• Example 2.3: The Quiz Problem. Consider a game where a
person is given two questions and must decide which question to
answer first
– Question 1 will be answered correctly with probability 0.8, and the
person will then receive as prize $100
– While question 2 will be answered correctly with probability 0.5, and the
person will then receive as prize $200
– If the first question attempted is answered incorrectly, the quiz
terminates
– Which question should be answered first to maximize the expected
value of the total prize money received?
X
 0 .2

PX  x    0 .8  0 .5
 0 .8  0 .5

, x0
 0 .5

PY  y    0 .2  0 .5
 0 .8  0 .5

, y0
Y
, x  100
, x  300
, y  200
E[ X ]  0 .2  0  0 .4  100
 0 .4  300  160
E[Y ]  0 .5  0  0.1  200
 0.4  300  140
, y  300
Probability-Berlin Chen 18
Recitation
• SECTION 2.4 Expectation, Mean, Variance
– Problems 18, 19, 21, 24
Probability-Berlin Chen 19