Download Joint distribution, and independence

Probability & Statistics
Professor Wei Zhu
July 22nd
(1) Let’s have some fun: A Fair Game or Not?
A gambler goes to bet. The dealer has 3 dice, which are fair, meaning that the
chance that each face shows up is exactly 1/6.
The dealer says: "You can choose your bet on a number, any number from 1 to
6. Then I'll roll the 3 dice. If none show the number you bet, you'll lose $1. If
one shows the number you bet, you'll win $1. If two or three dice show the
number you bet, you'll win $3 or $5, respectively."
Is this a fair game? (*A fair game is such that the expected, or equivalently the
long term average, winning is zero.)
𝟏
Solution: Let 𝑿 be # of dice show the number you bet, then 𝑿~𝑩𝒊𝒏𝒐𝒎𝒊𝒂𝒍(𝟑, 𝟔)
(Because the dealer roll 3 dice independently, and the probability for each
𝟏
dice show the number you bet is 𝟔. That is,
𝒙
𝟓 𝟑−𝒙
𝟑 𝟏
𝑷(𝑿 = 𝒙) = ( ) ( ) ( )
, 𝒙 = 𝟎, 𝟏, 𝟐, 𝟑
𝒙 𝟔
𝟔
Let 𝒀 be the amount of money you get, then,
𝟎
𝟓 𝟑−𝟎
𝟓 𝟑 𝟏𝟐𝟓
𝟑 𝟏
𝑷(𝒀 = −𝟏) = 𝑷(𝑿 = 𝟎) = ( ) ( ) ( )
=( ) =
𝟎 𝟔
𝟔
𝟔
𝟐𝟏𝟔
𝟏
𝟓 𝟑−𝟏
𝟏 𝟓 𝟐
𝟕𝟓
𝟑 𝟏
𝑷(𝒀 = 𝟏) = 𝑷(𝑿 = 𝟏) = ( ) ( ) ( )
=𝟑∙ ∙( ) =
𝟏 𝟔
𝟔
𝟔 𝟔
𝟐𝟏𝟔
𝟐
𝟓 𝟑−𝟐
𝟏 𝟐 𝟓
𝟏𝟓
𝟑 𝟏
𝑷(𝒀 = 𝟑) = 𝑷(𝑿 = 𝟐) = ( ) ( ) ( )
=𝟑∙( ) ∙ =
𝟐 𝟔
𝟔
𝟔
𝟔 𝟐𝟏𝟔
𝟑
𝟓 𝟑−𝟑
𝟏 𝟑
𝟏
𝟑 𝟏
𝑷(𝒀 = 𝟓) = 𝑷(𝑿 = 𝟑) = ( ) ( ) ( )
=( ) =
𝟑 𝟔
𝟔
𝟔
𝟐𝟏𝟔
Thus, your expected winning is
𝑬(𝒀) = ∑ 𝒚 ∙ 𝑷(𝒀 = 𝒚) = (−𝟏) ∙
𝒚
𝟏𝟐𝟓
𝟕𝟓
𝟏𝟓
𝟏
+𝟏∙
+𝟑∙
+𝟓∙
=𝟎
𝟐𝟏𝟔
𝟐𝟏𝟔
𝟐𝟏𝟔
𝟐𝟏𝟔
Therefore, it is a fair game.
1
(2) *Review: The derivative and integral of some important functions one
should remember.
𝑑 𝑘
(𝑥 ) = 𝑘𝑥 𝑘−1
𝑑𝑥
𝑑 𝑥
(𝑒 ) = 𝑒 𝑥
𝑑𝑥
𝑑
1
(ln 𝑥) =
𝑑𝑥
𝑥
𝑥=𝑏
𝑏
1
1
∫ 𝑥 𝑑𝑥 = (
𝑥 𝑘+1 )|
=
(𝑏 𝑘+1 − 𝑎𝑘+1 )
𝑘
+
1
𝑘
+
1
𝑎
𝑥=𝑎
𝑘
𝑥=𝑏
𝑏
𝑥
𝑥
= 𝑒𝑏 − 𝑒𝑎
∫ 𝑒 𝑑𝑥 = 𝑒 |
𝑎
𝑥=𝑎
*The Chain Rule
𝑑
𝑔[𝑓(𝑥)] = 𝑔′[𝑓(𝑥)] ∙ 𝑓′(𝑥)
𝑑𝑥
𝑑
2
2
For example: 𝑑𝑥 (𝑒 𝑥 ) = 𝑒 𝑥 ∙ 2𝑥
*The Product Rule
𝑑
[𝑔(𝑥) ∙ 𝑓(𝑥)] = 𝑔′ (𝑥)𝑓(𝑥) + 𝑔(𝑥)𝑓′(𝑥)
𝑑𝑥
*To Find the Maximum/Minimum of a (continuous & differentiable) Function
e. g. g(𝑥) = (𝑥 − 1)2 , 𝑥 ∈ [0, 2]
We first set the first derivative to zero, find the solutions, then compare to the
boundary values. This shall yield the minimum and maximum. We see that
𝑑
𝑑𝑥
g(𝑥) = 2(𝑥 − 1) = 0 → 𝑥 = 1.
Soon we realize that at this point we have the minimum, while the maximum is
achieved at the two boundary points.
2
(3)
Normal Distribution
Q. Who invented the normal distribution?
* Left: Abraham de Moivre (26 May 1667 in Vitry-le-François, Champagne,
France – 27 November 1754 in London, England) *From the Wikipedia
* Right: Johann Carl Friedrich Gauss (30 April 1777 – 23 February 1855)
<i> Probability Density Function (p.d.f.)
X ~ N (  ,  2 ) : X follows normal distribution of mean  and variance  2
f ( x) 
1
2 

e
( x )2
2 2
,    x  , x  R
3
b
P(a  X  b)   f ( x)dx = area under the pdf curve bounded by a and b
a
<ii> Cumulative Distribution Function (c.d.f.)
x
F ( x)  P( X  x)   f (t )dt

f ( x)  [ F ( x)]' 
d
F ( x)
dx
(4). Mathematical Expectation (Review).

Continuous random variable:
E[ g ( X )] 
 g ( x) f ( x)dx

Discrete random variable:
all
E[ g ( X )]   g ( x) P( X  x)
Properties of Expectations:
(1) E(c) = c, where c is a constant
4
(2) E[c*g(X)] = c*E[g(X)], where c is a constant
(3) E[g(X)+h(Y))]= E[g(X)]+E[h(Y)], for any X&Y
(4) E[g(X)*h(Y)] = E[g(X)]*E[h(Y)], if X & Y are independent –otherwise it is usually
not true.
Special case:

1) (population) Mean:   E ( X )   x  f ( x)dx

Note: E(aX+b) =aE(X)+b, where a & b are constants
2) (population) Variance:

Var(X) =  2  E[( X   )2 ]   ( x   )2  f ( x)dx  E( X 2 )  [ E( X )]2

Note: Var(aX+b) = a2Var(X), where a & b are constants
3) Moment generating function:

M X (t )  E (e tX )   e tx f ( x)dx , when X is continuous.

𝑴𝑿 (𝒕) = 𝐄(𝐞𝐭𝐗 ) = ∑𝐚𝐥𝐥 𝐩𝐨𝐬𝐬𝐢𝐛𝐥𝐞 𝐯𝐚𝐥𝐮𝐞𝐬 𝐨𝐟 𝐗 𝐞𝐭𝐱 𝐟(𝐱)
For normal distribution, X ~ N ( ,  ), f ( x) 
2

M X (t )   e tx f ( x)dx  e
1
2 

e
( x )2
2 2
,   x  
1
2
t   2t 2

4) Moment:
1st (population) moment: 𝐸(𝑋) = ∫ 𝑥 ∙ 𝑓(𝑥) 𝑑𝑥
2nd (population) moment: 𝐸(𝑋 2 ) = ∫ 𝑥 2 ∙ 𝑓(𝑥) 𝑑𝑥
…
Kth (population) moment: 𝐸(𝑋 𝑘 ) = ∫ 𝑥 𝑘 ∙ 𝑓(𝑥) 𝑑𝑥
5
Theorem. If X 1 , X 2 are independent, then
M X 1  X 2 (t )  M X1 (t ) M X 2 (t )
Theorem. Under regularity conditions, there is 1-1 correspondence between
the pdf and the mgf of a given random variable X. That is,
11
pdf f ( x) 
mgf M X (t ) .
e.g. Sgt. Jones wishes to select one army recruit into his unit. It is known that the IQ
distribution for all recruits is normal with mean 180 and standard deviation 10.
What is the chance that Sgt. Jones would select a recruit with an IQ of at least 200?
Sol) Let X represents the IQ of a randomly selected recruit.
X ~ N (   180,   10)
P( X  200)  P(
X  180 200  180

)  P( Z  2)  2.28%
10
10
e.g. Sgt. Jones wishes to select three army recruits into his unit. It is known that the
IQ distribution for all recruits is normal with mean 180 and standard deviation 10.
What is the chance that Sgt. Jones would select three recruits with an average IQ of
at least 200?
Sol) Let 𝑋1, 𝑋2 , 𝑋3 represents the IQ of three randomly selected recruits.
𝑋𝑖
𝑖.𝑖.𝑑.
~
𝑁(𝜇, 𝜎 2 ),
𝑖 = 1, 2, 3
X ~ N (  *  180, * 
P( X  200)  P(
10
)
3
X  180 200  180

)  P( Z  2 3 )  0.03%
10 / 3
10 / 3
6
*(5). Joint distribution, and independence
Definition. The joint cdf of two random variables X and Y are defined as:
𝐹𝑋,𝑌 (𝑥, 𝑦) = 𝐹(𝑥, 𝑦) = 𝑃(𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦)
Definition. The joint pdf of two discrete random variables X and Y are defined as:
𝑓𝑋,𝑌 (𝑥, 𝑦) = 𝑓(𝑥, 𝑦) = 𝑃(𝑋 = 𝑥, 𝑌 = 𝑦)
Definition. The joint pdf of two continuous random variables X and Y are defined
as:
𝑓𝑋,𝑌 (𝑥, 𝑦) = 𝑓(𝑥, 𝑦) =
𝜕2
𝐹(𝑥, 𝑦)
𝜕𝑥𝜕𝑦
Definition. The marginal pdf of the discrete random variable X or Y can be obtained
by summation of their joint pdf as the following:
𝑓𝑋 (𝑥) = ∑𝑦 𝑓(𝑥, 𝑦) ; 𝑓𝑌 (𝑦) = ∑𝑥 𝑓(𝑥, 𝑦) ;
Definition. The marginal pdf of the continuous random variable X or Y can be
obtained by integration of the joint pdf as the following:
∞
∞
𝑓𝑋 (𝑥) = ∫−∞ 𝑓(𝑥, 𝑦) 𝑑𝑦; 𝑓𝑌 (𝑦) = ∫−∞ 𝑓(𝑥, 𝑦) 𝑑𝑥;
Definition. The conditional pdf of a random variable X or Y is defined as:
𝑓(𝑥|𝑦) =
𝑓(𝑥, 𝑦)
𝑓(𝑥, 𝑦)
; 𝑓(𝑦|𝑥) =
𝑓(𝑦)
𝑓(𝑥)
Definition. The joint moment generating function of two random variables X and Y
is defined as
𝑀𝑋,𝑌 (𝑡1 , 𝑡2 ) = 𝐸(𝑒 𝑡1 𝑋+𝑡2 𝑌 )
7
Note that we can obtain the marginal mgf for X or Y as follows:
𝑀𝑋 (𝑡1 ) = 𝑀𝑋,𝑌 (𝑡1 , 0) = 𝐸(𝑒 𝑡1 𝑋+0∗𝑌 ) = 𝐸(𝑒 𝑡1 𝑋 ); 𝑀𝑌 (𝑡2 ) = 𝑀𝑋,𝑌 (0, 𝑡2 ) = 𝐸(𝑒 0∗𝑋+𝑡2 ∗𝑌 )
= 𝐸(𝑒 𝑡2 ∗𝑌 )
Theorem. Two random variables X and Y are independent ⇔ (if and only if)
𝐹𝑋,𝑌 (𝑥, 𝑦) = 𝐹𝑋 (𝑥)𝐹𝑌 (𝑦) ⇔ 𝑓𝑋,𝑌 (𝑥, 𝑦) = 𝑓𝑋 (𝑥)𝑓𝑌 (𝑦) ⇔ 𝑀𝑋,𝑌 (𝑡1 , 𝑡2 ) = 𝑀𝑋 (𝑡1 ) 𝑀𝑌 (𝑡2 )
Definition. The covariance of two random variables X and Y is defined as
𝐶𝑂𝑉(𝑋, 𝑌) = 𝐸[(𝑋 − 𝜇𝑋 )(𝑌 − 𝜇𝑌 )].
Theorem. If two random variables X and Y are independent, then we have
𝐶𝑂𝑉(𝑋, 𝑌) = 0. (*Note: However, 𝐶𝑂𝑉(𝑋, 𝑌) = 0 does not necessarily mean that X
and Y are independent.)
Homework #2, Due Friday, July 25, before our lecture
𝐐𝟏. 𝐿𝑒𝑡 𝑋1 & 𝑋2 𝑏𝑒 𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡 𝑁(𝜇, 𝜎 2 ). Prove that 𝑋1 + 𝑋2 and 𝑋1 − 𝑋2 are
independent.
Q2. 𝐿𝑒𝑡 𝑍 𝑏𝑒 𝑁(0,1), (1) please derive the covariance between 𝑍 𝑎𝑛𝑑 𝑍 2 ; (2) Are
𝑍 𝑎𝑛𝑑 𝑍 2 independent?
Q3. Let ~𝑁(3, 4), please calculate 𝑃(1 < 𝑋 < 3)
Q4. Jack & Jill plan to meet at the gate of the famous Los Angeles Chinese Theater
(LACT) between 12noon and 1pm tomorrow. The one who arrives first will wait for
the other for 30 minutes, and then leave. What is the chance that the two friends will
be able to meet at LACT during their appointed time period assuming that each one
will arrive at LACT independently, each at a random time between 12noon and 1pm?
8