Download Document

Statistics (I)
Joint Probability Distributions
Learning Objectives
1.
Define two discrete random variables
2.
Describe two continuous random variables
3.
Discuss covariance and correlation
4.
Define bivariate normal Distribution
5.
Describe linear combination of random variables
6.
Discuss Chebyshev’s inequality
Why Two Discrete Random
Variables ?
To find out the simultaneous behavior of two
random variables.
 Example:
X and Y are random variables defined for
the two dimensions of a part of a production
line. What is the probability that X is within the
range of 2.95 ~ 3.05 and Y is within the range of
7.60~7.80?

Two Discrete
Random Variables


If X and Y are discrete random variables, the
joint distribution of X and Y is a description of
the set of points (x, y) in the range of (X, Y)
along with the probability of each point.
It is sometimes referred to as the bivariate
probability distribution or bivariate
distribution.
Two Discrete
Random Variables

Definition

The joint probability mass function of the discrete
random variable X and Y, denoted as fXY(x,y) ,
satisfies
1. f XY ( x, y )  0
2.  f XY (x,y)  1
x
y
3. f XY (x,y)  P( X  x, Y  y )
Two Discrete Random Variables --Example


A financial company uses X=1, 2,3 to
represent low, medium, and high income
customers respectively. Also they use
Y=1,2,3,4 to represent mutual funds, bonds,
stocks, and options respectively.
Then the joint probability of X and Y could be
represented by following table.
Two Discrete Random Variables --Example
Y=1 Y=2 Y=3 Y=4
X=1 0.1
0
0
0
X=2 0.2
0.1
0.2
0
X=3 0.1
0
0.1
0.2
(1) f XY ( x , y )  0
(2)

x y
f XY ( x , y )  1
Two Discrete Random Variables --Example
0.2
0.15
0.1
0.05
Y
0
1
2
3
4
1
2
3
Marginal Probability
Distribution (1/2)
f X ( x )  P( X  x )   f XY ( x, y )
Rx
fY ( y )  P(Y  y )   f XY ( x, y )
Ry
Marginal Probability
Distribution (2/2)
where
Rx denotes the set of all points in the range of
(X, Y) for which X=x and
Ry denotes the set of all points in the range of
(X, Y) for which Y=y
Mean of Marginal
Probability Distribution

If the marginal probability distribution of X
has the probability mass function fX(x), then
Example(1/3)

A financial company use X=1,2,3 to
represent low, medium, and high income
customers respectively. Also they use
Y=1,2,3,4 to represent mutual funds,
bonds, stocks, and options respectively.

Then the joint probability of X and Y could
be represented by following table.
Example(2/3)
Y=1
Y=2
Y=3
Y=4
X=1
0.1
0
0
0
X=2
0.2
0.1
0.2
0
X=3
0.1
0
0.1
0.2
Example(3/3)

Please find marginal probability of fX(x,y),
fY(x,y), and their mean and variance.
Solution to Example(1/3)
Solution to Example(2/3)
Solution to Example(3/3)
Y=1 Y=2
Y=3
Y=4
fX(x)
X=1
0.1
0
0
0
0.1
X=2
0.2
0.1
0.2
0
0.5
X=3
0.1
0
0.1
0.2
0.4
0.1
0.3
0.2
1
fY(Y) 0.4
Conditional Probability
Distribution

Given discrete random variable X and Y
with joint probability mass function
fXY(x, y) the conditional probability mass
function of Y given X = x is
Properties of Conditional
Probability Distribution
Example(1/3)

Same example mentioned above
Y=1
Y=2
Y=3
Y=4
X=1
0.1
0
0
0
X=2
0.2
0.1
0.2
0
X=3
0.1
0
0.1
0.2
Example(2/3)

Computed results
Y=1
Y=2
Y=3
Y=4
fX(x)
X=1
0.1
0
0
0
0.1
X=2
0.2
0.1
0.2
0
0.5
X=3
0.1
0
0.1
0.2
0.4
fY(Y)
0.4
0.1
0.3
0.2
1
Example(3/3)

The conditional probability of Y given X =
2 is
Properties of Conditional
Probability Distribution
Conditional mean of Y given X = x
Conditional variance of Y given X = x
Example
Please find E(X|y=1)
Example
Please find V(X|y=1).
Independence of Two Discrete
Random Variables(1/2)
 For discrete random variables X and Y, if
any one of the following properties is
true, then the others are also true, and X
and Y are independent .
Independence of Two Discrete
Random Variables(2/2)
(1) f XY ( x , y )  f X ( x ) f Y ( y ) for all x and y
( 2 ) f Y | x ( y )  f Y ( y ) for all x and y with f X ( x ) > 0
( 3) f X | y ( x )  f X ( x )
for all x and y with f Y ( y ) > 0
( 4) P ( X  x, Y  y )  f X ( x ) f Y ( y )
for all ( x , y ) in the range of X and Y
Example
In above example, are X and Y independent ?
Solution to Example
P(X=1, Y=1)=0.1
fX(Y=1)=0.4
fY(X=1)=0.1
P(X=1, Y=1) ≠ fX(Y=1)*fY(X=1)
Thus X and Y are dependent!
Two Continuous
Random Variables

A joint probability density function for the
continuous random variables X and Y, denoted
as fXY(x, y), satisfied the following properties.
Example
 Suppose X is the time to failure of a
component, and Y is also the time to failure
of its spare part. If the joint probability
density function of X and Y is:
Please find P(X≦1000, Y≦2000)=?
Solution to Example
0.915 Why?
Marginal Probability
Distributions

If the joint probability density function of continuous
random variables X and Y is fXY (x,y), then the
probability density functions of X and Y are
Rx denotes the set of all points in the range of (X,Y) for which X=x
and
Ry denotes the set of all points in the range of (X,Y) for which Y=y
Example
From above example, please find P(Y>2000)=?
Solution to Example
0.05 Why?
Conditional Probability
Distribution
Given continuous random variable X and Y
with joint probability density function
fXY(x, y) the conditional probability density
function of Y given X = x is
f Y |x
(x, y) 
for all
f XY ( x , y )
fX (x)
fX (x) > 0
Properties of Conditional
Probability Distribution
Example
From above example, please find the
conditional density function Y given that
X=x.
Solution to Example
p.225
Properties of Conditional
Probability Distribution
Conditional mean of Y given X = x
Conditional variance of Y given X = x
Independence of Two Continuous
Random Variables (1/2)

For continuous random variables X
and Y, if any one of the following
properties is true, then the others are
also true, and X and Y are
independent .
Independence of Two continuous
Random Variables(2/2)
(1) f XY ( x , y )  f X ( x ) f Y ( y ) for all x and y
( 2 ) f Y | x ( y )  f Y ( y ) for all x and y with f X ( x ) > 0
( 3) f X | y ( x )  f X ( x )
for all x and y with f Y ( y ) > 0
( 4 ) P ( X A, Y  B)  P ( X A) P(Y  B)
for any sets A and B in the range of X
and Y, respectively.
Covariance and Correlation
Covariance

Expected value of a function of two random
variables h(x, y)
R  h( x, y ) f XY ( x, y )
X,Y discre te


E[h( X , Y )]   h( x, y ) f ( x, y)dxdy X,Y continuous 
XY



R

Covariance
The covariance between the random variables
X and Y, denoted as cov(X,Y) or σXY ,
 XY  E[( X   X )(Y  Y )]  E( XY )   X Y
Example(1/3)
A financial company use X=1,2,3 to
represent low, medium, and high
income customers respectively. Also
they use Y=1,2,3,4 to represent mutual
funds, bonds, stocks, and options
respectively.
Then the joint probability of X and Y could be
represented by following table.
Example(2/3)
Y=1
Y=2
Y=3
Y=4
X=1
0.1
0
0
0
X=2
0.2
0.1
0.2
0
X=3
0.1
0
0.1
0.2
Example(3/3)
Please find the covariance of X and Y.
Solution to Example
 XY  E ( XY )   X Y
 11 0.1  2 1 0.2  2  2  0.1
 2  3  0.2  3 1 0.1  3  3  0.1
 3  4  0.2  2.3  2.3
 5.7  (2.3)
 0.41
2
Correlation
The correlation between the random variables X and
Y, denoted as XY ,
ρ XY
cov( X , Y )
σ XY


V ( X )V (Y ) σ Xσ Y
whe re - 1  ρ XY  1
Correlation
If X and Y are independent random variables,
then
σ XY  ρ XY  0
Example
From the above example, please find the
correlation of X and Y.
Solution to Example
ρ XY
cov( X , Y )
σ XY


V ( X )V (Y ) σ Xσ Y
0.41

(0.41)(1.41)
Bivariate Normal Distribution
Bivariate Normal
Distribution
The probability density function of a
bivariate normal distribution
f XY ( x, y;σ X ,σ Y ,μ X ,μ Y ,ρ )

1
2σ Xσ Y
 - 1  ( x  μ X ) 2 2ρ (x- X )( y  μ Y ) ( y  μ Y ) 2  
exp 



2 
2
2
2
σ Xσ Y
σ Y 
1 ρ
 2(1 -ρ )  σ X
for -∞<x<∞ and -∞<y<∞, with parameters σX >0,
σy >0 , -∞<μX<∞ and -∞< μY <∞, and -1<ρ<1
Bivariate Normal
Distribution
If X and Y have a bivariate normal distribution
with joint probability density fXY(x,y; σX, σY, μX, μY,
ρ), then the marginal probability distributions of X
and Y are normal with means μX and μY, and
standard deviations σX and σY, respectively.
Bivariate Normal
Distribution
If X and Y have a bivariate normal distribution
with joint probability density fXY(x,y; σX, σY, μX, μY,
ρ), then the correlation between X and Y is ρ
Bivariate Normal
Distribution
If X and Y have a bivariate normal
distribution with ρ=0, then X and Y are
independent.
Linear Combinations of Random
Variables
Linear Combinations of
Random Variables
Given random variables X1, X 2, …., Xp and
constants c1, c 2, …., cp , then
Y= c1 X1+ c 2 X 2 …., cp Xp
is a linear combination of X1, X 2, …., Xp .
Linear Combinations of
Random Variables
If Y= c1 X1+ c 2 X 2 …., cp Xp , then
E(Y)= c1 E(X1)+ c 2 E(X 2)…., cp E(Xp)
Linear Combinations of
Random Variables
If Y= c1 X1+ c 2 X 2 …., cp Xp , then
V (Y )  c V ( X 1 )  c V ( X 2 )  ...  c V ( X p )
2
1
2
2
2
p
 2  ci c j cov( X i , X j )
i j 2
Furthermore, if X1, X 2, …., Xp , are
independent, then
V (Y )  c12V ( X1 )  c22V ( X 2 )  ...  c 2pV ( X p )
Linear Combinations of
Random Variables
If X  ( X1  X2 ... X p ) / p with E(Xi)=μ for i=1,2,…,p,
then
E( X )  μ
Furthermore, if X1, X 2, …., Xp , are also
independent, with V(Xi ) σ 2 for i=1,2,…,p, then
V (X ) σ 2 / p
Reproductive Property
of the Normal Distribution
If X1, X 2, …., Xp , are independent, normal
2
random variables with E(Xi)=μi and V(Xi)=σi , for
i=1,2,..,p, then
Y= c1 X1+ c 2 X 2 …., cp Xp
is a normal random variable with
E(Y)= c1 E(X1)+ c 2 E(X 2)…., cp E(Xp) and
V (Y )  c12 21  c22 2 2  ...  c 2p 2 p
Chebyshev’s Inequality
Chebyshev’s Inequality
For any random variable X with mean μ and
variance σ2 ,
P(|X- μ| ≧ c σ) ≦ 1/c 2
for c >0
Example
The process of drilling holes in PCB produces
diameters with a standard deviation of 0.01
millimeter. How many diameters must be
measured so that the probability is at least 8/9
that the average of the measured diameters is
within 0.005 of the process mean.
Solution to Example(1/2)
Let X1, X2, …, Xn be the independent random
variables that denote the diameters of n holes.
So the average measured diameter is :
X  ( X 1  X 2  ...  X n ) / n
 E ( X )   and V ( X )  0.01 / n
2
 σ X  0.01 / n
2
Solution to Example(2/2)
P(|X- μ| ≧ c σ) ≦ 1/c 2
So
P (| X   | c(0.012 / n)1/ 2 )  1 / c 2
let c  3
P (| X   | 3(0.012 / n)1/ 2 )  1 / 9
i.e. P (| X   | 3(0.01 / n)
2
So 3(0.012 / n)1/ 2  0.005
and n  36
1/ 2
)  8/9