Download 3. Random Variables

3. Random Variables
(Fig.3.1)
1
Random Variables
If A  X 1 ( B) belongs to the associated field F, then the
probability of A is well defined .
in that case we can say :
Probabilit y of the event " X ( )  B "  P( X 1 ( B)). (3-1)
Random Variable (r.v): A finite single valued
function X (  ) that maps the set of all experimental outcomes 
into the set of real numbers R is said to be a r.v, if the set   | X ( )  x 
is an event ( F )for every x in R.
2
Random Variables
X :r.v, X 1 ( B)  F
B represents semi-infinite intervals of the form {  x  a}
 The Borel collection B of such subsets of R is the
smallest -field of subsets of R that includes all semiinfinite intervals of the above form.
 if X is a r.v, then
(3-2)
 | X ( )  x    X  x 

is an event for every x.
3
Distribution Function
P | X ( )  x   FX ( x)  0.
(3-3)
The role of the subscript X in (3-3) is only to identify the
actual r.v.
FX (x)
is said to the Probability Distribution
Function associated with the r.v X.
4
Properties of a PDF
if g(x) is a distribution function, then
(i) g ( )  1, g ( )  0,
(ii) if x1  x2 , then g ( x1 )  g ( x2 ),

(iii) g ( x )  g ( x ),for all x.
(3-4)
5
Properties of a PDF
Suppose FX (x) defined in (3-3)
(i) FX ()  P | X ( )     P()  1
(3-5)
(3-6)
and FX ()  P | X ( )     P( )  0.
(ii) I f x1  x2 , then the subset (, x1 )  (, x2 ).
Consequently the event  | X ( )  x1   | X ( )  x2 ,
since X ( )  x1 implies X ( )  x2 . As a result
FX ( x1 )  P X ( )  x1   P X ( )  x2   FX ( x2 ),
(3-7)
=>the probability distribution function is nonneg ative
and monotone nondecreasing.
6
Properties of a PDF
(iii) Let x  xn  xn1    x2  x1, and consider the event
since
Ak   | x  X ( )  xk .
(3-8)
 x  X ( )  xk   X ( )  x   X ( )  xk ,
(3-9)
using mutually exclusive property of events we get
P( Ak )  Px  X ( )  xk   FX ( xk )  FX ( x).
(3-10)
But  Ak 1  Ak  Ak 1 , and hence

lim Ak   Ak   and hence lim P( Ak )  0.
k 
k 1
k 
(3-11)
7
Properties of a PDF
Thus
lim P( Ak )  lim FX ( xk )  FX ( x)  0.
k 
k 

lim
x

x
, the right limit of x, and hence
But k  k
FX ( x  )  FX ( x),
(3-12)
i.e., FX (x) is right-continuous, justifying all properties of
a distribution function.
8
Additional Properties of a PDF
(iv) If FX ( x0 )  0 for some x0 , then FX ( x )  0, x  x0 .
(3-13)
This follows, since FX ( x0 )  P X ( )  x0   0 implies X ( )  x0 
is the null set, and for any x  x0 ,  X ( )  x  will be a
subset of the null set.
(v) P X ( )  x   1  FX ( x).
(3-14)
We have  X ( )  x   X ( )  x   
(vi) P x1  X ( )  x2   FX ( x2 )  FX ( x1 ), x2  x1.
(3-15)
The events X ( )  x1  and {x1  X ( )  x2} are mutually
exclusive and their union represents the event  X ( )  x2 .
9
Additional Properties of a PDF
(vii)
Let
P X ( )  x   FX ( x)  FX ( x  ).
x1  x   ,   0,
and x2  x. From (3-15)
lim P x    X ( )  x   FX ( x )  lim FX ( x   ),
 0
 0
(3-16)
(3-17)
or P X ( )  x   FX ( x)  FX ( x  ).
(3-18)
Thus the only discontinuities of a distribution function FX (x)
occur at points x0 where
P X ( )  x0   FX ( x0 )  FX ( x0 )  0.
(3-19)
is satisfied.
10
continuous-type & discrete-type r.v

X is said to be a continuous-type r.v if FX ( x )  FX ( x)
And from P X ( )  x0   FX ( x0 )  FX ( x0 )  0.

=>
 If F
PX  x  0.
(x) is constant except for a finite number of jump
discontinuities (piece-wise constant; step-type), then
X is said to be a discrete-type r.v. Ifxi is such a
discontinuity point, then
X
pi  PX  xi   FX ( xi )  FX ( xi ).
(3-20)
11
Example

Example 3.1: X is a r.v such that X ( )  c,   .Find FX (x).
Solution: For x  c,  X ( )  x    , so that FX ( x)  0,
and for x  c, X ( )  x  , so that FX ( x)  1. (Fig.3.2)
FX (x)
1
c
x
Fig. 3.2
at a point of discontinuity we get
P X  c   FX (c)  FX (c  )  1  0  1.
12
Example
Example 3.2: Toss a coin.   H ,T .Suppose the r.v X is
such that X (T )  0, X ( H )  1. Find FX (x).
Solution: For x  0,  X ( )  x    , so that FX ( x)  0.
0  x  1,  X ( )  x    T , so that FX ( x)  P T   1  p,
x  1,  X ( )  x    H , T   , so that FX ( x)  1. (Fig. 3.3)
FX (x)
1
q
1
x
Fig.3.3
at a point of discontinuity we get
P X  0   FX (0)  FX (0 )  q  0  q.
13
Example
Example:3.3 A fair coin is tossed twice, and let the r.v X
represent the number of heads. Find FX (x).
Solution: In this case
X ( HH )  2, X ( HT )  1, X (TH )  1, X (TT )  0.
x  0,
X ( )  x    FX ( x)  0,
1
0  x  1, X ( )  x   TT   FX ( x)  P TT   P(T ) P(T )  ,
4
3
1  x  2, X ( )  x   TT , HT , TH   FX ( x)  P TT , HT , TH   ,
4
x  2, X ( )  x    FX ( x)  1. (Fig. 3.4)
14
Example
FX (x)
1
3/ 4
1/ 4
1
x
2
Fig. 3.4
From Fig.3.4,
PX  1  FX (1)  FX (1 )  3 / 4  1 / 4  1 / 2.
15
Probability density function (p.d.f)
dFX ( x )
f X ( x) 
.
dx
(3-21)
Since
dFX ( x )
FX ( x  x )  FX ( x )
 lim
 0,
x 0
dx
x
from the monotone-nondecreasing nature of FX (x),
=>
f X ( x)  0
for all x
16
Probability density function (p.d.f)

f X (x) will be a continuous function, if X is a

continuous type r.v
if X is a discrete type r.v as in (3-20), then its p.d.f
f (x)
has the general form (Fig. 3.5)
X
f X ( x)   pi ( x  xi ),

xi
i
pi
xi
x
Fig. 3.5
represent the jump-discontinuity points in FX (x).
As Fig. 3.5 shows f X (x) represents a collection of
positive discrete masses, and it is known as the
probability mass function (p.m.f ) in the discrete
case.
17
Probability density function (p.d.f)
x
From (3-23),
FX ( x)   f X (u)du.

Since FX ()  1,
=> 


f X ( x)dx  1,
P x1  X ( )  x2   FX ( x2 )  FX ( x1 )   f X ( x )dx. (3-22)
x2
x1
the area under f X (x) in the interval ( x1, x2 ) represents
the probability in (3-22).
FX (x)
f X (x)
1
x1 x2
(a)
x
x1 x2
(b)
x
18
Continuous-type random variables
1. Normal (Gaussian):
f X ( x) 
1
2
FX ( x )  
x

2
e
( x   ) 2 / 2 2
1
2 2
e
(3-23)
.
( y   ) 2 / 2 2
x
dy  G
,
  
1 y /2
Where G ( x )  
e
dy

2
the notation X  N (  , 2 ) will be used to represent (3-23).
x
2
f X (x)

Fig. 3.7
x
19
Continuous-type random variables
2. Uniform:X U (a, b),
a  b,
(Fig. 3.8)
 1
, a  x  b,
f X ( x)   b  a
 0, otherwise.
1
ba
(3.24)
f X (x)
a
b
x
Fig. 3.8
20
Continuous-type random variables
2. Exponential: X   ( )
(Fig. 3.9)
1 x / 

 e
, x  0,
f X ( x)   

 0, otherwise.
f X (x)
(3.25)
note : X  
x
Fig. 3.9
21
Exponential distribution





Assume the occurences of nonoverlapping
intervals are independent, and assume:
q(t): the probability that in a time interval t no
event has occurred.
x: the waiting time to the first arrival
Then we have: P(x>t)=q(t)
t1 and t2 : two consecutive nonoverlapping
intervals,
22
Exponential distribution


Then we have: q(t1) q(t2) = q(t1+t2)
The only bounded solution is:
q(t )  e  t
Hence
FX (t )  P( X  t )  1  q(t )  1  e t
So the pdf is exponential.
If
the occurrences of events over nonoverlapping intervals are
independent, the corresponding pdf has to be exponential.
23
Memoryless property of exponential distribution
Let s, t  0
. Consider the events
{x  t  s} and {x  s}. Then

P{ X  t  s}
P{ X  t  s | X  s} 
P{ X  s}

e
( t  s )
s
e
(since {X  t  s}  {X  s})
 e t  P{ X  t}
24
Continuous-type random variables
x
4. Gamma: X  G( ,  ) if (  0,   0) (Fig. 3.10)
 x 1
x / 

e
, x  0,

f X ( x )   ( ) 

0, otherwise.

( ) 

 1  x
x
 e dx
(3-26)
f X (x)
0
If
  n an integer
(n )  (n  1)!.
Fig. 3.10
25
Continuous-type random variables


The exponential random variable is a special case of
gamma distribution with   1
The  2 (chi-square) random variable with n degrees
of freedom is a special case of gamma distribution
with
  n / 2 and   2
26
Continuous-type random variables
5. Beta: X   (a, b) if (a  0, b  0) (Fig. 3.11)
1

x a 1 (1  x )b1 , 0  x  1,

f X ( x )    ( a , b)

0,
otherwise.
(3-27)

where the Beta function  (a, b) is defined as
 ( a, b) 

1

0
f X (x )
0
x
1
Fig. 3.11
u a 1 (1  u ) b 1 du.
( a )(b)
 ( a  b)
(3-28)
Beta distribution with a=b=1 is the uniform distribution on
(0,1).
27
6. Chi-Square: X   2 (n), if (Fig. 3.12)
1

x n / 21e  x / 2 , x  0,
 n/2
f X ( x )   2 ( n / 2 )
(3-29)

0,
otherwise.

f X (x )
x
Fig. 3.12
Note that  2 (n) is the same as Gamma (n / 2, 2).
7. Rayleigh: X  R( 2 ), if (Fig. 3.13)
2
2
x

 2 e  x / 2 , x  0,
f X ( x )  

 0, otherwise.
f X (x )
(3-30)
x
Fig. 3.13
8. Nakagami – m distribution:
 2  m m 2 m 1  mx 2 / 
x e
, x0



f X ( x )   ( m )   

0
otherwise

(3-31)
28
f X (x )
9. Cauchy: X  C ( ,  ), if (Fig. 3.14)
f X ( x) 
 /
  (x  )
2
2
,    x  .
x

(3-32)
Fig. 3.14
10. Laplace: (Fig. 3.15)
f X ( x) 
1 |x|/ 
e
,    x  .
2
(3-33)
11. Student’s t-distribution with n degrees of freedom (Fig 3.16)
f T (t ) 
( n  1) / 2  
t 
1  
n
n ( n / 2) 
2
f X ( x)
 ( n 1) / 2
,    t  .
fT ( t )
x
Fig. 3.15
(3-34)
t
Fig. 3.16
29
12. Fisher’s F-distribution
{( m  n ) / 2} m m / 2 n n / 2
z m / 2 1
, z0

(mn) / 2
f z ( z)  
( m / 2) ( n / 2)
( n  mz )
(3-35)

0
otherwise

30
Discrete-type random variables
1. Bernoulli: X takes the values (0,1), and
P( X  0)  q,
P( X  1)  p.
2. Binomial: X  B(n, p), if (Fig. 3.17)
(3-36)
(3-37)
 n  k n k
P( X  k )  
, k  0,1,2,, n.
k 
p q
 
P( X  k )
12
n
k
The probability of k
successes in n experiments
with replacement (in ball
drawing)
Fig. 3.17
31
Discrete-type random variables
3. Poisson: X  P( ) , if (Fig. 3.18)
P( X  k )  e

k
k!
, k  0,1,2,, .
(3-38)
P( X  k )
k
Fig. 3.18
32
Discrete-type random variables

Poisson distribution represents the number of
occurrences of a rare event in a large number of trials.
Pk  P( X  k )
Pk 1 e  k 1 /( k  1)! k


 k
Pk

e  / k!

Pk Increasing with k from 0 to λ and decreasing after
that.
33
4. Hypergeometric:
P( X  k ) 
m
 
k 
 
 N m 


 n k 


,
N 
 
n 
 
max(0, m  n  N )  k  min( m, n )
(3-39)
The probability of k successes in n experiments without
replacement (ball drawing)
5. Geometric: X  g ( p ) if
P( X  k )  pqk , k  0,1,2,, ,
q  1  p.
(3-40)
34
6. Negative Binomial: X ~ NB (r, p), if
 k  1 r k  r
P( X  k )  
p q ,

 r 1
k  r, r  1,
.
(3-41)
7. Discrete-Uniform:
1
P( X  k ) 
, k  1,2,, N .
N
(3-42)
35