Download Document

Joint Probability Distribution
The joint probability distribution
function of X and Y is denoted by fXY(x,y).
The marginal probability distribution
function of X , fX(x) is obtained by
i. summing the probabilities corresponding
to each y value and the given x. (discrete
case)
fX(x)= SY fXY(x,y)|X=x
ii. Integrating out Y from the joint pdf
(continuous case)
fX(x)= Y fXY(x,y)dy
The conditional probability distribution
function of X given Y is denoted by fX|Y(x|y)
fX|Y(x|y)=fX, Y(x,y)/ fY(y)
[We similarly define fY|X(y|x)]
Random variables X and Y are
independent if and only if
fX|Y(x|y)=fX(x) for all x and y
Joint Probability Distribution
Covariance of two random variables X
and Y Cov(X,Y)
 E{(X- mX)(Y-mY)}
 E(XY- Y mX–X mY +mXmY)
 E(XY) - mXE(Y) –mY E(X) + mXmY
 E(XY)–mXmY- mXmY + mXmY
 E(XY)- mXmY
The Coefficient of Correlation between
two random variables X and Y
s(X,Y)  Cov(X,Y)/ sXsY
Two random variables X and Y are
uncorrelated if
s(X,Y) = 0; or if
E(XY)=mXmY
An important result:
Suppose that X and Y are two random
variables that have mean mX, mY and
standard deviation sX and sY
respectively.
Then for Z  aX +bY
E(Z)  amX + bmY
VarZ  a2VarX + b2VarY + 2abcov(X,Y)
 a2s2X+ b2 s2Y + 2absXY sX sY
VarZ  a2VarX + b2VarY + 2abcov(X,Y)
 a2s2X+ b2 s2Y + 2absXY sX sY
where sXY  Coefficient of Correlation
between X and Y.
If sXY = -1 then
VarZ = (asX -bsY)2
Joint Probability Distribution
Recall: Two random variables x
and y are independent if their joint p.d.f.
fXY (x,y) is the product of the respective
marginal p.d.f. fX(x) and fY(y). That is,
fXY (x,y) = fX(x). fy(y)
Theorem:
Independence of two random
variables X and Y imply that they are
uncorrelated
(but the converse is not
always true)
Proof:
E(XY) = xy fXY(x,y)dxdy
E(XY) = xy g(x)h(y)dxdy
E(XY) = (x g(x)dx)y h(y)dy
E(XY) = (mX)y h(y)dy
E(XY) = mX y h(y)dy
E(XY) = mXmY
The Normal Distribution
A continuous distribution with the pdf:
f(X) = {1/(s2p) }e –1/2[(X-mx)/sx)2 ]
For the Standard Normal Distribution
Variable Z, s = 1; m = 0
f(z) = {1/2p }e –(1/2) z2
Suppose that X and Y are two random
variables such that they have mean mX,
mY and standard deviation sX and sY
respectively.
Also assume that both X and Y are
normally distributed.
Then if W  aX +bY
W ~ Normal(mw, s2w) with
mw  amX + bmY and
s2w  a2s2X+ b2 s2X + 2absXY sX sY
where sXY is the relevant correlation
coefficient.
Message: A linear combination of two
or more normally distributed
(independent or not) r.v.s has
a normal distribution as well.
The c2distribution:
Consider Z ~ Normal(0,1). Consider
Y = Z2.
Then Y has a c2 distribution of 1
degree of freedom (d.o.f.).
We write it as Y ~ c2(1) .
Consider Z1, Z2, …Zn independent
random variables each ~ Normal(0,1)
Then their sum SZi2 has a c2
distribution with d.o.f. = n.
That is,
SZi2 ~ c2(n)
Consider two independent random
variables X ~ Normal(0,1) and
Y ~ c2(n)
Then the variable w  X/ (Y/n) has a
t-distribution with d.o.f. = n.
That is,
w ~ t(n)
An Application
Consider X ~ Normal(m, s2). Then
XMEAN ~ Normal (m, s2/n)
Consider X ~ (m, s2). Then
XMEAN ~ Normal (m, s2/n) if n is
‘large’ (CLT)
Then the variable w  (XMEAN –
m)/s/n), has a t-distribution with
d.o.f. = n-1
where s is an unbiased estimator of
s
Suppose that X ~ c2(m). and Y ~ c2(n)
and the variables X and Y are
independent.
Then the variable v ≡ (x/m) /(y/n)
has an F distribution with the
numerator d.o.f. = m and the
denominator d.o.f = n.
V ~ Fm,n
Suppose that X ~ c2(1). and Y ~ c2(n)
and the variables X and Y are
independent.
Then the variable v ≡ x /(y/n) has an
F distribution with the numerator
d.o.f. = 1 and the denominator d.o.f
= n. V ~ F1,n
Clearly, V ~ t(n)