Download Document

Strong law of large numbers
Let X1, X2, ..., Xn be a set of independent random variables
having a common distribution, and let E[Xi] = m. then, with
probability 1
X1  X1  ...  X n
 m as n  .
n
Central Limit Theorem
Let X1, X2, ..., Xn be a set of independent random variables
having a common distribution with mean m and variance s.
Then the distribution of
X 1  X 1  ...  X n  nm
s n
tends to the standard normal as n  . That is
P(
X 1  X 1  ...  X n  nm
as n  .
s n
 a) 
1
2

a

e
 x2 / 2
dx
Conditional probability and
conditional expectations
Let X and Y be two discrete random variables, then the
conditional probability mass function of X given that Y=y is
defined as
P{ X  x, Y  y} p( x, y)
pX |Y ( x | y)  P{ X  x | Y  y} 

.
P{Y  y}
p( y )
for all values of y for which P(Y=y)>0.
Conditional probability and
conditional expectations
Let X and Y be two discrete random variables, then the
conditional probability mass function of X given that Y=y is
defined as
P{ X  x, Y  y} p( x, y)
pX |Y ( x | y)  P{ X  x | Y  y} 

.
P{Y  y}
p( y )
for all values of y for which P(Y=y)>0.
The conditional expectation of X given that Y=y is defined as
E[ X | Y  y ]   xP{ X  x | Y  y}   xp X |Y ( x | y ).
x
x
Let X and Y be two continuous random variables, then the
conditional probability density function of X given that Y=y
is defined as
f ( x, y)
f X |Y ( x | y) 
.
fY ( y )
for all values of y for which fY(y)>0.
Let X and Y be two continuous random variables, then the
conditional probability density function of X given that Y=y
is defined as
f ( x, y)
f X |Y ( x | y) 
.
fY ( y )
for all values of y for which fY(y)>0.
The conditional expectation of X given that Y=y is defined as

E[ X | Y  y]   xf X |Y ( x | y)dx.

E[ X ]  E[ E[ X | Y  y ]]  E[ E[ X | Y ]]
E[ X ]   E[ X | Y  y ]P(Y  y ) if Y is discrete
y

E[ X ]   E[ X | Y  y ] f ( y )dy if Y is continuous.

Proof
 E[ X | Y  y]P(Y  y)  { xP( X  x | Y  y)}P(Y  y)
y
y
x
Proof
 E[ X | Y  y]P(Y  y)  { xP( X  x | Y  y )}P(Y  y )
y
y
x
P ( X  x, Y  y )
P(Y  y )
  x
P(Y  y )
x
y
Proof
 E[ X | Y  y]P(Y  y)  { xP( X  x | Y  y)}P(Y  y)
y
y
x
P ( X  x, Y  y )
  x
P(Y  y )
P (Y  y )
y
x
  xP ( X  x, Y  y )
y
x
Proof
 E[ X | Y  y]P(Y  y)  { xP( X  x | Y  y )}P(Y  y )
y
y
x
P ( X  x, Y  y )
  x
P(Y  y )
P(Y  y )
y
x
  xP( X  x, Y  y )
y
x
  xP( X  x, Y  y )
x
y
Proof
 E[ X | Y  y]P(Y  y)  { xP( X  x | Y  y )}P(Y  y )
y
y
x
P ( X  x, Y  y )
  x
P(Y  y )
P(Y  y )
y
x
  xP( X  x, Y  y )
y
x
  xP( X  x, Y  y )
x
y
  xP( X  x)
x
Proof
 E[ X | Y  y]P(Y  y)  { xP( X  x | Y  y )}P(Y  y )
y
y
x
P ( X  x, Y  y )
  x
P(Y  y )
P(Y  y )
y
x
  xP( X  x, Y  y )
y
x
  xP( X  x, Y  y )
x
y
  xP( X  x)
x
 E[ X ]
The sum of a random number of
random variables
Example: The number N of customers that place orders each
day with an online bookstore is a random variable with
expected value E[N].
The sum of a random number of
random variables
Example: The number N of customers that place orders each
day with an online bookstore is a random variable with
expected value E[N]. The number of books Xi that each
customer i (i = 1, 2, ..., N) purchases is also a random variable
E[Xi] with expected value E[Xi].
The sum of a random number of
random variables
Example: The number N of customers that place orders each
day with an online bookstore is a random variable with
expected value E[N]. The number of books Xi that each
customer i (i = 1, 2, ..., N) purchases is also a random variable
E[Xi] with expected value E[Xi]. What is the expected value of
the total number of books Y sold each day? What is its
variance?
The sum of a random number of
random variables
Example: The number N of customers that place orders each
day with an online bookstore is a random variable with
expected value E[N]. The number of books Xi that each
customer i (i = 1, 2, ..., N) purchases is also a random variable
E[Xi] with expected value E[Xi]. What is the expected value of
the total number of books Y sold each day? What is its
variance? Assume that the number of books are independent
and identically distributed with the same mean E[Xi]=E[X] and
variance Var[Xi]=E[X] for i=1,..., N. Also assume the number of
books purchased per customer is independent of the total
number of customers.
The expected value
E[Y ]  E[ i 1 X i ]  E[ E[ i 1 X i | N  n]]
N
N
Since E[ i 1 X i | N  n]  E[ i 1 X i ]  nE[ X ],
N
E[Y ]  E[nE[ X ]]  E[ X ]E[ N ]
n
The variance

 Var[Y ]  E 


N
i 1
 
2
Xi
N




E
X

i

 i 1 

2
The variance
 X  
 

X    E  E  

 

 Var[Y ]  E 


 E


N
i 1
i 1
2
i
2
N
i
N




E
X

i

 i 1 
N
i 1
Xi

2

2

| N  n  ,

The variance
 X  

 


X    E  E   X  | N  n   ,


 



X  | N  n   E   X  




 Var[Y ]  E 



 E  


 E

N
i 1
N
i 1
2
N
i 1
2
i
N




E
X

i

 i 1 
2
N
i 1
i
2
i
2
2
n
i 1
i
i
2
 Var   i 1 X i   E   i 1 X i   nVar  X   n 2 E[ X ]2




n
n
The variance
 X  

 


X    E  E   X  | N  n   ,


 



X  | N  n   E   X  




 Var[Y ]  E 



 E  


 E

i 1
2
N
i 1
2
N
i

 N X i 

E

 i 1 
2
N
i 1
i
i
2
N
i 1
2
2
n
i 1
i
i
2
 Var   i 1 X i   E   i 1 X i   nVar  X   n 2 E[ X ]2




n
 
 E E 
 

 i 1 X i
N
n

2

| N  n    E[ N ]Var  X   E[ N 2 ]E[ X ]2

The variance
 X  

 


X    E  E   X  | N  n   ,


 



X  | N  n   E   X  




 Var[Y ]  E 



 E  


 E

i 1
2
N
i 1
2
N
i
N




E
X

i

 i 1 
2
N
i 1
i
i
2
N
i 1
2
2
n
i 1
i
i
 Var   i 1 X i   E   i 1 X i   nVar  X   n 2 E[ X ]2




2
N
 

 E  E   i 1 X i | N  n    E[ N ]Var  X   E[ N 2 ]E[ X ]2

 
n


 E


n
2

 i 1 X i
N

2

2
2

E
[
N
]Var
X

E
[
N
]
E
[
X
]




 Var(Y )  E 


N
i 1
 
2
Xi
N




E
X

i

 i 1 

2
= E[ N ]Var( X )  E[ N ]E[ X ]   E[ N ]E[ X ] 
2
2

 E[ N ]Var( X )  E[ X ]2 E[ N 2 ]  E[ N ]2
=E[ N ]Var( X )  E[ X ]2 Var ( N )

2
If N is Poisson distributed with parameter , the random
Y = X1+X2+...+ XN is called a compound Poisson random
variable
Var(Y )  E[ N ]Var( X )  E[ X ]2 Var ( N )
= Var( X )  E[ X ]2 
= E[ X 2 ]
Computing probabilities by conditioning
Let E denote some event. Define a random variable X by
1, if E occurs
X 
0, if E does not occur
 E[ X ]  P ( E )
Computing probabilities by conditioning
Let E denote some event. Define a random variable X by
1, if E occurs
X 
0, if E does not occur
 E[ X ]  P( E )
 E[ X | Y  y ]   xP ( X  x | Y  y )  P ( X  1| Y  y )  P ( E | Y  y )
x
Computing probabilities by conditioning
Let E denote some event. Define a random variable X by
1, if E occurs
X 
0, if E does not occur
 E[ X ]  P ( E )
 E[ X | Y  y ]   xP ( X  x | Y  y )  P ( X  1| Y  y )  P ( E | Y  y )
x
 P ( E )  E[ X ]  E[ E[ X | Y  y ]]
=  P ( E | Y  y )P (Y  y )   if Y is discrete
y

  P ( E | Y  y ) fY ( y )dy   if Y is continuous

Example 1: Let X and Y be two independent continuous
random variables with densities fX and fY. What is P(X<Y)?
Example 1: Let X and Y be two independent continuous
random variables with densities fX and fY. What is P(X<Y)?

P( X  Y )   P( X  Y | Y  y ) fY ( y ) dy

Example 1: Let X and Y be two independent continuous
random variables with densities fX and fY. What is P(X<Y)?

P( X  Y )   P( X  Y | Y  y ) fY ( y)dy


  P ( X  y ) fY ( y )dy

Example 1: Let X and Y be two independent continuous
random variables with densities fX and fY. What is P(X<Y)?

P( X  Y )   P( X  Y | Y  y ) fY ( y ) dy


  P ( X  y ) fY ( y )dy


  FX ( y ) fY ( y )dy

Example 2: Let X and Y be two independent continuous
random variables with densities fX and fY. What is the
distribution of X+Y?
Example 2: Let X and Y be two independent continuous
random variables with densities fX and fY. What is the
distribution of X+Y?
FX Y (a )  P( X  Y  a )
Example 2: Let X and Y be two independent continuous
random variables with densities fX and fY. What is the
distribution of X+Y?

FX Y (a)  P( X  Y  a)   P( X  Y  a | Y  y) fY ( y)dy

Example 2: Let X and Y be two independent continuous
random variables with densities fX and fY. What is the
distribution of X+Y?

FX Y (a )  P( X  Y  a )   P ( X  Y  a | Y  y ) fY ( y )dy


  P ( X  y  a ) fY ( y )dy

Example 2: Let X and Y be two independent continuous
random variables with densities fX and fY. What is the
distribution of X+Y?

FX Y (a)  P( X  Y  a)   P( X  Y  a | Y  y ) fY ( y )dy


  P ( X  y  a ) fY ( y )dy


  P ( X  a  y ) fY ( y )dy

Example 2: Let X and Y be two independent continuous
random variables with densities fX and fY. What is the
distribution of X+Y?

FX Y (a)  P( X  Y  a)   P( X  Y  a | Y  y ) fY ( y )dy


  P ( X  y  a) fY ( y )dy


  P ( X  a  y ) fY ( y )dy


  FX (a  y ) fY ( y )dy

Example 2: Let X and Y be two independent continuous
random variables with densities fX and fY. What is the
distribution of X+Y?

FX Y (a)  P( X  Y  a)   P( X  Y  a | Y  y ) fY ( y )dy


  P ( X  y  a) fY ( y )dy


  P ( X  a  y ) fY ( y )dy


  FX (a  y ) fY ( y )dy

Example 3: (Thinning of a Poisson) Suppose X is a