Download Sums of Random Variables

Sums of Random Variables
Probability Density of a Sum of
Random Variables
Let Z = X + Y . Then for Z to be less than z , X must be less
than z − Y . Therefore, the CDF for Z is
FZ ( z ) =
∞ z− y
∫ ∫ f ( x, y ) dxdy
XY
−∞ −∞
If X and Y are independent,
⎛ z− y
⎞
FZ ( z ) = ∫ fY ( y ) ⎜ ∫ f X ( x ) dx ⎟dy
−∞
⎝ −∞
⎠
and it can be shown that
∞
fZ ( z ) =
∞
∫ f ( y ) f ( z − y ) dy = f ( z ) ∗ f ( z ) ← Convolution
Y
−∞
X
Y
X
Moment Generating Functions
The moment-generating function Φ X ( s ) of a CV random variable
X is defined by Φ X ( s ) = E ( e sX ) =
∞
∫
f X ( x ) e sx dx.
−∞
Φ X ( s ) = L ⎡⎣f X ( x ) ⎤⎦ s →− s
where L ⎡⎣f X ( x ) ⎤⎦ is the Laplace transform of f X ( x )
∞
d
Φ X ( s ) ) = ∫ f X ( x ) xe sx dx
(
ds
−∞
∞
⎡d
⎤
= ∫ x f X ( x ) dx = E ( X )
⎢⎣ ds ( Φ X ( s ) ) ⎥⎦
s →0
−∞
n
⎡
⎤
d
n
E ( X ) = ⎢ n ( Φ X ( s ) )⎥
⎣ ds
⎦ s →0
Moment Generating Functions
Example
Let X and Y be exponential random variables with PDF's
⎧λ X e − λ X x , x > 0
⎧λY e − λY y , y > 0
fX ( x) = ⎨
and fY ( y ) = ⎨
, otherwise
, otherwise
⎩0
⎩0
Find the PDF of Z = X + Y .
f Z ( z ) = f X ( z ) ∗ fY ( z )
The moment-generating function of Z is Φ Z ( s ) = Φ X ( s ) ΦY ( s ) .
ΦX (s) =
λX
λX − s
, ΦY ( s ) =
λY
λY − s
⇒ ΦZ ( s) =
λX
λY
λ X − s λY − s
Moment Generating Functions
Example
⎛ λX
λY ⎞
f Z ( z ) = L ( Φ X ( − s ) ΦY ( − s ) ) = L ⎜
⎟
+
λ
+
λ
s
s
⎝
X
Y ⎠
−1
λ X λY
⎛ λ X λY
⎜
λ X − λY
−1 λY − λ X
fZ ( z ) = L ⎜
+
s + λY
⎜ s + λX
⎜
⎝
−1
⎞
⎟
⎟
⎟
⎟
⎠
λ X λY − λY z
⎧ λ X λY − λX z
, z>0
e
e
+
⎪
λX − λY
f Z ( z ) = ⎨ λY − λX
⎪0
, otherwise
⎩
Moment Generating Functions
The moment-generating function Φ X ( s ) of a DV random variable
X is defined by Φ X ( s ) = E ( e sX ) =
⎡d
⎤
= E(X )
⎢⎣ ds Φ X ( s ) ⎥⎦
s →0
∞
∑
n =−∞
P ( X = n ) e sn =
∑
x∈S X
n
⎡
⎤
d
n
E ( X ) = ⎢ n ( Φ X ( s ) )⎥
⎣ ds
⎦ s →0
PX ( x ) e sx .
Moment Generating Functions
Example
Let X be a Poisson random variable. Its moment-generating function is
(
)
α e s −1
ΦX (s) = e
. Find its mean and variance.
s
⎡d
⎤
s α ( e −1) ⎤
⎡
= αe e
=α
E ( X ) = ⎢ Φ X ( s )⎥
⎥⎦ s →0
⎣ ds
⎦ s →0 ⎢⎣
2
s
s
⎡
⎤
d
s α ( e −1) ⎤
2
2 2 s α ( e −1)
⎡
= α e e
E ( X ) = ⎢ 2 Φ X ( s )⎥
+αe e
= α 2 +α
⎥⎦ s →0
⎣ ds
⎦ s →0 ⎢⎣
Var ( X ) = α 2 + α − α 2 = α
The Central Limit Theorem
If N independent random variables are added to form a resultant
N
random variable Z = ∑ X n then
n =1
f Z ( z ) = f X1 ( z ) ∗ f X 2 ( z ) ∗ f X 2 ( z ) ∗" ∗ f X N ( z )
and it can be shown that, under very general conditions, the pdf
of a sum of a large number of independent random variables
with continuous pdf’s approaches a Gaussian pdf regardless of
the shapes of the individual pdf’s.
The Central Limit Theorem
Multivariate Probability Density
FX1 , X 2 ,", X N ( x1 , x2 ," , xN ) ≡ P ( X 1 ≤ x1 ∩ X 2 ≤ x2 ∩ " ∩ X N ≤ xN )
0 ≤ FX1 , X 2 ,", X N ( x1 , x2 ," , xN ) ≤ 1 , − ∞ < x1 < ∞ , " , − ∞ < xN < ∞
FX1 , X 2 ,", X N ( −∞," , −∞ ) = FX1 , X 2 ,", X N ( −∞," , xk ," , −∞ )
= FX1 , X 2 ,", X N ( x1 ," , −∞," , xN ) = 0
FX1 , X 2 ,", X N ( +∞," , +∞ ) = 1
FX1 , X 2 ,", X N ( x1 , x2 ," , xN ) does not decrease
if any number of x ' s increase
FX1 , X 2 ,", X N ( +∞," , xk ," , +∞ ) = FX k ( xk )
Multivariate Probability Density
f X1 , X 2 ,", X N ( x1 , x2 ," , xN ) =
∂N
∂ x1∂ x2 " ∂xN
FX1 , X 2 ,", X N ( x1 , x2 ," , xN )
f X1 , X 2 ,", X N ( x1 , x2 ," , xN ) ≥ 0 , − ∞ < x1 < ∞ , " , − ∞ < xN < ∞
∞
∞ ∞
∫"∫ ∫ f
−∞
−∞ −∞
X1 , X 2 ,", X N
FX1 , X 2 ,", X N ( x1 , x2 ," , xN ) =
f X k ( xk ) =
∞
−∞ −∞
x2 x1
∫"∫ ∫ f
−∞
∞ ∞
∫"∫ ∫ f
−∞
xN
( x1 , x2 ," , xN ) dx1dx2 " dxN
X1 , X 2 ,", X N
−∞ −∞
X1 , X 2 ,", X N
=1
( λ1 , λ2 ," , λN ) d λ1d λ2 " d λN
( x1 , x2 ," , xk −1 , xk +1 ," , xN ) dx1dx2 " dxk −1dxk +1 " dxN
P {( X 1 , X 2 ," , X N ) ∈ R} = ∫ " ∫∫ f X1 , X 2 ,", X N ( x1 , x2 ," , xN ) dx1dx2 " dxN
E ( g ( X 1 , X 2 ," , X N ) ) =
R
∞ ∞
∫ ∫ g ( x , x ," , x ) f
1
−∞ −∞
2
N
X1 , X 2 ,", X N
( x1 , x2 ," , xN ) dx1dx2 " dxN