Download Chapter 6 More than one variable

Chapter 6
More than one variable
6.1
Bivariate discrete distributions
Suppose that the r.v.’s X and Y are discrete and take on the values xj and yj , j ≥ 1,
respectively. Then the joint p.d.f. of X and Y , to be denoted by fX,Y , is defined
by: fX,Y (xj , yj ) = P (X = xj , Y = yj ) and fX,Y (x, y) = 0 when (x, y) 6= (xj , yj )
(i.e., at least one of x or y is not equal to xj or yj , respectively).
The marginal distribution of X is defined by the probability function
X
P (X = xi ) =
P (X = xi , Y = yj ).
j
P (Y = yj ) =
X
P (X = xi , Y = yj ).
i
P
Note that P (X = xi ) ≥ 0 and i P (X = xi ) = 1. The mean and variance of
X can be defined in the usual way.
The conditional distribution of X given Y = yj is defined by the probability
function
P (X = xi , Y = yj )
.
P (X = xi |Y = yj ) =
P (Y = yj )
The conditional mean of X given Y = yj is defined by
X
E[X|Y = yj ] =
xi P (X|Y = yj ).
i
and similarly for the variance:
V ar[X|Y = yj ] = E(X 2 |Y = yj ) − (E(X|Y = yj ))2 .
Although the E[X|Y = yj ] depends on the particular values of Y , it turns out
that its average does not, and, indeed, is the same as the E[X]. More precisely, it
holds:
E[E(X|Y )] = E[X] and E[E(Y |X)] = E[Y ].
1
CHAPTER 6. MORE THAN ONE VARIABLE
2
That is, the expectation of the conditional expectation of X is equal to its
expectation, and likewise for Y.
The covariance of X and Y is defined by
cov[X, Y ] = E[(X − E[X])(Y − E[Y ])] = E[XY ] − E[X]E[Y ]
where
E[XY ] =
XX
i
xi yj P (X = xi , Y = yj ).
j
The result obtained next provides the range of values of the covariance of two
r.v.s; it is also referred to as a version of the CauchySchwarz inequality.
Theorem 6.1 Cauchy Schwarz inequality
1. Consider the r.v.s X and Y with E[X] = E[Y ] = 0 and V ar[X] = V ar[Y ] =
1. Then always −1 ≤ E[XY ] ≤ 1, and E[XY ] = 1 if and only if P (X =
Y ) = 1, and E[XY ] = −1 if and only if P (X = −Y ) = 1.
2
and
2. For any r.v.s X and Y with finite expectations and positive variances σX
2
σY , it always holds: −σX σY ≤ Cov(X, Y ) ≤ σX σY , and Cov(X, Y ) = σX σY
if and only if P [Y = E[Y ] + σσXY (X − E[X])] = 1, Cov(X, Y ) = −σX σY if and
only if P [Y = E[Y ] − σσXY (X − EX)] = 1.
The correlation coefficient between X and Y is defined by
Y −E[Y ]
]
]
] = Cov[X,Y
= E[XY σ]−E[X]E[Y
.
corr[X, Y ] = E[ X−E[X]
σX
σY
σX σY
X σY
The correlation always lies between −1 and +1.
Example 6.1
Let X and Y be two r.v.s with finite expectations and equal (finite) variances,
and set U = X + Y and V = X − Y . Calculate if r.v.s U and V are correlated.
Solution
E[U V ] = E[(X + Y )(X − Y )] = E(X 2 − Y 2 ) = E[X 2 ] − E[Y 2 ]
E[U ]E[V ] = [E(X + Y )][E(X − Y )] = (E[X] + E[Y ])(E[X] − E[Y ]) = (E[X])2 −
(E[Y ])2
Cov(U, V ) = E[U V ] − E[U ]E[V ] = (E[X 2 ] − E[X]2 ) − (E[Y 2 ] − E[Y ]2 )
= V ar(X) − V ar(Y ) = 0
U and V are uncorrelated.
△
For two r.v.s X and Y with finite expectations, and (positive) standard deviations σX and σY , it holds:
CHAPTER 6. MORE THAN ONE VARIABLE
3
2
V ar(X + Y ) = σX
+ σY2 + 2Cov(X, Y )
and
2
V ar(X + Y ) = σX
+ σY2
if X and Y are uncorrelated.
Proof
V ar(X + Y ) = E[(X + Y ) − E(X + Y )]2 = E[(X − E[X]) + E(Y − E[Y ])]2
= E(X − E[X])2 + E(Y − E[Y ])2 + 2E[(X − E[X])(Y − E[Y ])]
2
= σX
+ σY2 + 2Cov(X, Y ).
△
Random variables X and Y are said to be independent if
P (X = xi , Y = yj ) = P (X = xi )P (Y = yj )
.
If X and Y are independent then Cov[X, Y ] = 0. The converse is NOT true.
There exist many pairs of random variables with Cov[X, Y ] = 0 which are not
independent.
Example 6.2
A fair dice is thrown three times. The result of first throw is scored as X1 = 1
if the dice shows 5 or 6 and X1 = 0 otherwise; X2 and X3 are scored likewise for
the second and third throws.
Let Y1 = X1 + X2 and Y2 = X1 − X3 .
4
Show that P (Y1 = 0, Y2 = −1) = 27
. Calculate the remaining probabilities in
the bivariate distribution of the pair (Y1 , Y2 ) and display the joint probabilities in
an appropriate table.
1. Find the marginal probability distributions of Y1 and Y2 .
2. Calculate the means and variances of Y1 and Y2 .
3. Calculate the covariance of Y1 and Y2 .
4. Find the conditional distribution of Y1 given Y2 = 0.
5. Find the conditional mean of Y1 given Y2 = 0.
CHAPTER 6. MORE THAN ONE VARIABLE
4
Solution
P (X1 = 1) = P ({5, 6}) = 13 , P (X2 = 1) = 31 , P (X3 ) = 13
For Y1 to be 0, X1 and X2 must be 0. Then Y2 to be −1, X3 must be 1.
4
P (Y1 = 0, Y2 = −1) = P (X1 = 0)P (X2 = 0)P (X3 = 1) = 23 23 13 = 27
0
Y2
-1
0
1
P
4
27
8
27
0
12
27
Y1
1
P
2
0
2
27
6
27
4
27
12
27
6
27
15
27
6
27
1
27
2
27
3
27
1
1. Marginal probability distribution of Y1 :
y1
P (Y1 = y1 )
0
1
2
12
27
12
27
3
27
Marginal probability distribution of Y2 :
y2
P (Y2 = y2 )
-1
0
-1
6
27
15
27
6
27
2.
12
3
2
12
+1×
+2×
=
27
27
27
3
12
12
3
8
02 ×
+ 12 ×
+ 22 ×
=
27
27
27
9
2
4
8
2
=
E[Y12 ] − (E[Y1 ])2 = −
9
3
9
6
15
6
−1 ×
+0×
+1×
=0
27
27
27
15
6
4
6
+ 02 ×
+ 12 ×
=
(−1)2 ×
27
27
27
9
4
2
2
E[Y2 ] − (E[Y2 ]) =
9
E[Y1 ] = 0 ×
E[Y12 ] =
V ar[Y1 ] =
E[Y2 ] =
E[Y22 ] =
V ar[Y2 ] =
2
4
3. Cov[Y1 , Y2 ] = E[Y1 Y2 ] − E[Y1 ]E[Y2 ] = E[Y1 Y2 ] = 1 × (−1) × 27
+ 1 × 1 × 27
+
2
2
2 × 1 × 27 = 9
5
CHAPTER 6. MORE THAN ONE VARIABLE
4.
P (Y1 = 0|Y2 = 0) =
P (Y1 = 0 ∩ Y2 = 0)
=
P (Y2 = 0)
P (Y1 = 1|Y2 = 0) =
P (Y1 = 1 ∩ Y2 = 0)
=
P (Y2 = 0)
P (Y1 = 2|Y2 = 0) =
P (Y1 = 2 ∩ Y2 = 0)
=
P (Y2 = 0)
8
27
15
27
6
27
15
27
1
27
15
27
=
8
,
15
=
6
15
=
1
15
5.
E[Y1 |Y2 = 0] = 1 ×
1
8
6
+2×
=
15
15
15
△
Exercises
Exercise 6.1
The random variables X and Y have a joint probability function given by
c(x2 y + x) x=-2,-1,0,1,2 y=1,2,3
f (x, y) =
0
otherwise
Determine the value of c.
Find P (X > 0) and P (X + Y = 0)
Find the marginal distributions of X and Y .
Find E[X] and V ar[X].
Find E[Y ] and V ar[Y ].
Find the conditional distribution of X given Y = 1 and E[X|Y = 1].
Find the probability function for Z = X + Y and show that E[Z] = E[X] + E[Y ]
Find Cov[X, Y ] and show that V ar[Z] = V ar[X] + V ar[Y ] + 2Cov[X, Y ].
Find the correlation between X and Y .
Are X and Y independent?
Solution
Table for the joined probabilities:
6
CHAPTER 6. MORE THAN ONE VARIABLE
X
0 1
2
0 2c 6c 10c
0 3c 10c 20c
0 4c 14c 30c
0 9c 30c 60c
probabilities must add to one, c =
-2 -1
1 2c 0
Y 2 6c c
3 10c 2c
18c 3c
1
Since the sum of
.
60
39
P (X > 0) = 60
P (X + Y = 0) = P (X = −2, Y = 2) + P (X = −1, Y = 1) =
1
10
Marginal distributions for X:
x
-2
-1
0
1
P (X = x) 18/60 3/60 0 9/60
Marginal distributions for Y :
2
30/60
1
2
3
y
P (Y = y) 10/60 20/60 30/60
E[X] = −2 ∗ 18/60 − 1 ∗ 3/60 + 0 ∗ 0 + 1 ∗ 9/60 + 2 ∗ 30/60 = 30/60 = 1/2
E[X 2 ] = (−2)2 ∗ 18/60 + (−1)2 ∗ 3/60 + 0 ∗ 0 + 12 ∗ 9/60 + 22 ∗ 30/60 = 3.4
V ar[X] = E[X 2 ] − E[X]2 = 3.4 − 0.52 = 3.15
E[Y ] = 1 ∗ 1/6 + 2 ∗ 1/3 + 3 ∗ 1/2 = 14/6 = 7/3
E[Y 2 ] = 12 ∗ 1/6 + 22 ∗ 1/3 + 32 ∗ 1/2 = 36/6 = 6.0
V ar[Y ] = 6.0 − (7/3)2 = 5/9
P (X = −2|Y = 1) = 0.2, P (X = −1|Y = 1) = 0, P (X = 0|Y = 1) = 0,
P (X = 1|Y = 1) = 0.2, P (X = 2|Y = 1) = 0.6
E[X|Y = 1] = −2 ∗ 0.2 − 1 ∗ 0 + 0 ∗ 0 + 1 ∗ 0.2 + 2 ∗ 0.6 = 1
Z =X +Y
z
-1
0
1
2
3
4
5
P (Z = z) 2/60 6/60 11/60 4/60 9/60 14/60 14/60
1
−1 ∗ 2 + 1 ∗ 11 + 2 ∗ 4 + 3 ∗ 9 + 4 ∗ 14 + 5 ∗ 14) = 170
E[Z] = 60
=
60
5
1
1
= 2 6 = 2 + 2 3 = E[X] + E[Y ]
E[X, Y ] = −2 ∗ 1 ∗ 2/60 − 2 ∗ 2 ∗ 6/60 − 2 ∗ 3 ∗ 10/60 − 1 ∗ 2 ∗ 1/60 − 1 ∗ 3 ∗ 2/60 +
1 ∗ 1 ∗ 2/60 + 1 ∗ 2 ∗ 3/60 + 1 ∗ 3 ∗ 4/60 + 2 ∗ 1 ∗ 6/60 + 2 ∗ 2 ∗ 10/60 + 2 ∗ 3 ∗ 14/60 = 1
Cov[X, Y ] = E[X, Y ] − E[X]E[Y ] = −1/6
1
E[Z 2 ] = 60
(1 ∗ 2 + 1 ∗ 11 + 4 ∗ 4 + 9 ∗ 9 + 16 ∗ 14 + 25 ∗ 14) = 684
60
V ar[Z] = 684
− 170
= 3.3722
60
60
V ar[X] + V ar[Y ] + 2Cov[X, Y ] = 3.15 + 5/9 − 2 ∗ 1/6 = 3.3722
corr[X, Y ] = √
Cov[X,Y ]
V ar[X]V ar[Y ]
= √ −1/6
3.15∗5/9
= −0.126
CHAPTER 6. MORE THAN ONE VARIABLE
X and Y are independent: P (X = −1, Y = 1) 6= P (X = −1)P (Y = 1).
7
△
Exercise 6.2
The following experiment is carried out. Three fair coins are tossed. Any coins
showing heads are removed and the remaining coins are tossed. Let X be the
number of heads on the first toss and Y the number of heads on the second toss.
Note that if X = 3 then Y = 0. Find the joint probability function and marginal
distributions of X and Y .
Solution
We have that P (Y = y, X = x) = P (Y = y|X = x)P (X = x).
Suppose X = 0, this has a probability 0.53 . Then Y |X = 0 has a Binomial distribution with parameters n = 3 and p = 0.5. Similarly Y |X = 1 has a Binomial
distribution with parameters n = 2 and p = 0.5. In this way we see we can produce
a table of the joint probabilities:
X
0
1
2
3
0 1/64 6/64 12/64 8/64 27/64
Y 1 5/64 12/64 12/64
0
27/64
0
0
9/64
2 3/64 6/64
3 1/64
0
0
0
1/64
1/8
3/8
3/8
1/8
1
Marginal distribution for X:
x
0
1
2
3
P (X = x) 1/8 3/8 3/8 1/8
Marginal distribution for Y :
0
1
2
y
P (Y = y) 27/64 27/64 9/64
3
1/64
△