Download SJSU EE102 Chan

EEE376F
Signals and Systems II
H Anthony Chan
[email protected]
http://www.eng.uct.ac.za/~achan
Mqhele Dlodlo
[email protected]
http://www.uct.ac.za/depts/staff/mdlodlo/
Department of Electrical Engineering
University of Cape Town
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 1 May 25, 2017
Multiple Random Variables
Joint Probability Mass Function
Joint Cumulative Distribution Function
Joint Probability Density Function
Study Bertsekas and Tsitsiklis
Do and understand exercises
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 2 May 25, 2017
Multiple Random Variables
 (X, Y) is called
 Bivariate random variable
 Joint random variable
 Random vector
 Similarly, X = (X1, X2, …, Xn) is a multivariate
random variable
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 3 May 25, 2017
Multiple Random Variables
 A Vector Random Variable X = (X1, X2, …, Xn) is a
function that maps each point s of the sample
space S to a vector of real numbers X1(s), X2(s), …,
Xn(s)
 Example of a vector random variable is the input
voltage and current to a circuit (Vin, Iin), which has
the value (Vin(t), Iin(t)) at time t
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 4 May 25, 2017
Multiple Random Variables
 A Random Variable X (or Y) is a function X(s) (or
Y(s)) that maps the points s (outcomes of the
experiment) of the sample space S to real numbers
 It is therefore a number associated with an experiment
 Each outcome of the experiment yields a sample point s,
which produces a sample value, say X(s) (or Y(s)), of the
random variable X (or Y)
 Two Random Variables X and Y jointly map the
point (or outcome) s to the point (X(s), Y(s)) in a 2dimensional plane
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 5 May 25, 2017
Random Variables
 Random Variables: X: S  SX and Y: S  SY
Sample Space: S
s1
s2 s
4
Y: S  SY
X: S  SX
x1 x2 x3 x4
Real Number, R
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
s3
y1
y2 y3
Real Number, R
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 6 May 25, 2017
Multiple Random Variables
 X: S1  SX
 Y: S2  SY
Sample Space: S1
s13
s11
s12
Sample Space: S2
x3
x1 x2
Real Number, R
y1 y2
Real Number, R
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
s21
If you want peace, work for Justice. (Pope John XXIII)
s22
Signals and Systems II
302 Page 7 May 25, 2017
Multiple Random Variables
 X: S1  SX; Y: S2  SY
Sample Space: S1
s11 s12
s13
Sample Space: S2
s22
s21
y2
y1
x1
x2
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
x3
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 8 May 25, 2017
Multiple Random Variables
 Vector Random Variables: (X,Y): S  (SX,SY )
Sample Space: S = S1S2
s11,s22 s12,s22 s13,s22
s12,s21
s11,s21
y2
y1
+
+
+ +
+ +
++
++
x1
x2
x3
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
s13,s21
+
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 9 May 25, 2017
Multiple Random Variables
 Event: E = {X1  E1}  {X2  E2}  …  {Xn  En}
 Probability: P[E]=P[{X1E1}{X2E2}…{XnEn}]

P[X1E1, X2E2, …, XnEn]
 Probability: P[E]=P[{sS  XE}]
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 10 May 25, 2017
Multiple Random Variables
Event
 X: S1  SX
 Y: S2  SY
Sample Space: S1
s13
s11 Event E1
s12 s14
Sample Space: S2
x3
x1 x2
Real Number, R
y1 y2
Real Number, R
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
s21 Event E2
s22
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 11 May 25, 2017
Multiple Random Variables
Event
 X: S  SX; Y: S  SY
Sample Space: S1
Event E1
s11 s12 s14 s13
Sample Space: S2
s22
Event E2
s21
y2
y1
x1
x2
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
x3
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 12 May 25, 2017
Multiple Random Variables
Event
 Vector Random Variables: (X,Y): S  (SX,SY )
E1
y2
y1
E2 s11,s22
s12,s22;s14,s22
s13,s22
s11,s21
s12,s21;s14,s21
s13,s21
+
+
+ +
+ +
++
++
x1
x2
x3
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
+
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 13 May 25, 2017
Independent Random Variables
 The random variables X1, X2, …, Xn are
independent means that for any events E1, E2, …,
En which involves only X1, X2, …, Xn respectively,
the probability of the joint event, P[E], is equal to
the product of the probabilities of the individual
events P[E1] P[E2] … P[En]:
 P[X1E1, …, XnEn] = P[X1E1] … P[XnEn]
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 14 May 25, 2017
Joint Probability Mass Function
Probability
Discrete: Continuous:
(xj,yk)
pX,Y(xj, yk)
(x,y)
Joint pmf at X=xj,Y=yk
Joint CDF for Xx,Yy FX,Y(xj, yk) FX,Y(x, y)
Joint pdf per unit area fX,Y(xj, yk) fX,Y(x, y)
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 15 May 25, 2017
Joint Probability Mass Function
 The joint probability mass function
 of the bivariate random variable (X, Y)
 at (xj, yk), j = 1, 2, …; k = 1, 2, …
 is equal to the probability
 of the event {X = xj}  {Y = yk}
 pX,Y(xj, yk) = P[{X = xj}  {Y = yk}]

 P[X = xj , Y = yk] j = 1, 2, …; k = 1, 2, …
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 16 May 25, 2017
Joint Probability Mass Function
 X,Y: S  RR
Sample Space: S = S1S2
s11,s22 s12,s22 s13,s22
s11,s21
+
y2
y1
+
s12,s21 s13,s21
+
+ +
+ +
+
++
x1
x3
x2
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
+
P(s11,s22) P(s12,s22) P(s13,s22)
P(s11,s21) P(s12,s21) P(s13,s21)
0
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 17 May 25, 2017
Joint Probability Mass Function
 X,Y: S  RR; pX,Y: RR  [0,1]
Sample Space: S = S1S2
s11,s22 s12,s22 s13,s22
s12,s21 s13,s21
s11,s21
+
y2
y1
+
+
+ +
+ +
+
++
x1
x3
x2
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
+
P(s11,s22) P(s12,s22) P(s13,s22)
P(s11,s21) P(s12,s21) P(s13,s21)
0
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 18 May 25, 2017
Joint Probability Mass Function
 pX,Y: RR  [0,1]
Sample Space: S = S1S2
s11,s22 s12,s22 s13,s22
s11,s21
+
y2
y1
+
s12,s21 s13,s21
+
+ +
+ +
+
++
x1
x3
x2
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
+
P(s11,s22) P(s12,s22) P(s13,s22)
P(s11,s21) P(s12,s21) P(s13,s21)
0
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 19 May 25, 2017
Joint Probability Mass Function
 pX,Y: RR  [0,1]
Sample Space: S = S1S2
s11,s22 s12,s22 s13,s22
s11,s21
s12,s21 s13,s21
P(s11,s21)
P(s12,s21)
P(s13,s21)
+
y2
y1
+
P(s11,s22)
+
+ +
+ +
++
++
x1
x3
x2
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
P(s12,s22)
P(s13,s22)
pX,Y: RR  [0,1]
If you want peace, work for Justice. (Pope John XXIII)
0
Signals and Systems II
302 Page 20 May 25, 2017
Example
Joint probability mass function
 Consider the following Joint pmf of (X, Y)
 pX,Y(0, 0) = 1/64
y
 pX,Y(1, 0) = 3/64
7/64 21/64 21/64 7/64
1
+
+
+
+
 pX,Y(2, 0) = 3/64
 pX,Y(3, 0) = 1/64
0 +1/64 +3/64 +3/64 +1/64x
 pX,Y(0, 1) = 7/64
0
1
2
3
 pX,Y(1, 1) = 21/64
 pX,Y(2, 1) = 21/64
 pX,Y(3, 1) = 7/64
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 21 May 25, 2017
Joint Probability Mass Function
 The probability of the event {(xj, yk)  E} is obtained
by the sum:
 P[{(xj, yk)  E}] = S S {(xj, yk)  E} pX,Y(xj, yk)

 Probability



  pX,Y(xj, yk) = 1
j 1 k 1
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 22 May 25, 2017
Marginal Probability Mass Function
 Marginal probability mass function
 pX(xj) = P[X = xj] = P[X = xj , Y = anything]

= P[ {X = xj , Y = y1}  {X = xj , Y = y2}  …]


=  pX,Y(xj, yk)
k 1
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 23 May 25, 2017
Example
Marginal probability mass function
 Marginal pmf: pX(xj) = Sk pX,Y(xj, yk)
 pX(0) = (1+7)/64 = 1/8
 pX(1) = (3+21)/64 = 3/8
 pX(2) = (3+21)/64 = 3/8
 pX(3) = (1+7)/64 = 1/8
y
1 +7/64 +21/64 +21/64 +7/64
0 +1/64 +3/64 +3/64 +1/64x
0
1
2
3
 pY(0) = (1+3+3+1)/64 = 1/8
 pY(1) = (7+21+21+7)/64 = 7/8
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 24 May 25, 2017
Joint Cumulative Distribution Function
Probability
Discrete: Continuous:
(xj,yk)
pX,Y(xj, yk)
(x,y)
Joint pmf at X=xj,Y=yk
Joint CDF for Xx,Yy FX,Y(xj, yk) FX,Y(x, y)
Joint pdf per unit area fX,Y(xj, yk) fX,Y(x, y)
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 25 May 25, 2017
Joint Cumulative Distribution Function
 The joint cumulative distribution function (CDF)
 of the bivariate random variable (X, Y)
 at (xj, yk), j = 1, 2, …; k = 1, 2, …
 is equal to the probability
y
 of the event {X  xj}  {Y  yk}
(xj, yk)
 FX,Y(xj, yk) = P[X  xj, Y  yk]
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
x
Signals and Systems II
302 Page 26 May 25, 2017
Joint Cumulative Distribution Function
 Properties from Probability:
 x2  x1 and y2  y1 implies FX,Y(x2, y2)  FX,Y(x1, y1)
 FX,Y(- , yk) = 0 = FX,Y(xj, - )
 FX,Y(, ) = 1
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 27 May 25, 2017
Example
Joint Cumulative Distribution Function
 Joint CDF of (X, Y) is FX,Y(x, y) = P[X  x, Y  y]
 FX,Y(1, - 0.1) = 0
y
 FX,Y(1, 0) = (1+3)/64
7/64 21/64 21/64 7/64
1
+
+
+
+
 FX,Y(0.1, 0.9) = 1/64
 FX,Y(3, 5) = 1
0 +1/64 +3/64 +3/64 +1/64x
 FX,Y(1.5, 2) = 32/64 = 1/2
0
1
2
3
 FX,Y(3.2, 0) = 8/64 = 1/8
 FX,Y(2.9, 0.9) = 7/64
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 28 May 25, 2017
Marginal Cumulative Distribution Function
y
 FX (x) = FX,Y(x, ) = P[X  x]
(xj, yk)
xj
 FY (y) = FX,Y(, y) = P[Y  y]
x
y
yk
x
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 29 May 25, 2017
Joint Probability Mass Function
Probability
Discrete: Continuous:
(xj,yk)
pX,Y(xj, yk)
(x,y)
Joint pmf at X=xj,Y=yk
Joint CDF for Xx,Yy FX,Y(xj, yk) FX,Y(x, y)
Joint pdf per unit area fX,Y(xj, yk) fX,Y(x, y)
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 30 May 25, 2017
Joint probability density function
 The joint probability density function (pdf)
 of the bivariate random variable (X, Y)
 is such that the probability of the event
y
 {(x, y)  E} is equal to the integral:
 P[(x, y)  E] =
 fX,Y(x’, y’) dx’ dy’
( x ', y ')E
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
x
Signals and Systems II
302 Page 31 May 25, 2017
Joint probability density function
Continuous random variable
 Shrink the event to E = (x≤x’≤x+δx, y≤y’≤y+δy):
y
y
x
x
 P[(x≤x’≤x+δx, y≤y’≤y+δy)] = fX,Y(x, y) δx δy
 fX,Y(x, y) is probability per unit area:δx δy
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 32 May 25, 2017
Joint probability density function
Joint Cumulative Distribution Function
 For the event E = (x’≤x, y’≤y), the integral is the joint
Cumulative Distribution Function
y
(x, y)
x y
 FX,Y(x, y) =   f X ,Y ( x' , y ' )dx' dy'
x
 
y
(x, y)
FX ,Y
  f X ,Y ( x, y ' )dy '
x

y
 FX ,Y ( x, y )
 f X ,Y ( x, y )
xy
2
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
x
y
(x, y)
x
Signals and Systems II
302 Page 33 May 25, 2017
Joint probability density function
Marginal Cumulative Density Function
 Marginal Cumulative Distribution Functions are:
 FX(x) =
y
x 
  f X ,Y ( x' , y )dx' dy
 
x
 y
 FY(y) =
  f X ,Y ( x, y ' )dxdy'
 
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
x
y
y
x
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 34 May 25, 2017
Marginal probability density function
d
f X ( x)  FX ( x)
dx
y
d x 

f X ,Y ( x ' , y )dx' dy


dx  

fY ( y ) 
x
x

 f X ,Y ( x, y )dy

y

y
 f X ,Y ( x, y )dx

What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
x
Signals and Systems II
302 Page 35 May 25, 2017
EEE376F
Signals and Systems II
If you want peace, work for Justice.
(Pope John XXIII)
© 2005
What I have is only borrowed from God so
that I may serve others. H Anthony Chan
If you want peace, work for Justice. (Pope John XXIII)
Signals and Systems II
302 Page 36 May 25, 2017