Download Continuous Random Variables

Continuous Distributions
Continuous random variables
For a continuous random variable X the probability
distribution is described by the probability density
function f(x), which has the following properties :
1.
f(x) ≥ 0

2.
 f  x  dx  1.

3.
b
P  a  X  b   f  x  dx.
a
Graph: Continuous Random Variable
probability density function, f(x)

 f  x  dx  1.

b
P  a  X  b   f  x  dx.
a
The Uniform distribution from a to b
Definition: A random variable , X, is said to have
a Uniform distribution from a to b if X is a
continuous random variable with probability
density function f(x):
 1

f  x  b  a
 0
a xb
otherwise
Graph: the Uniform Distribution
(from a to b)
0.4
f  x
0.3
0.2


1 

ba 

0.1
0
0
5
10
a
b
x
15
The Cumulative Distribution function, F(x)
(Uniform Distribution from a to b)
xa
 0
xa

F  x   P  X  x  
b  a
 1
0.4
f  x
0.3
a xb
xb
0.2
F  x
0.1
0
0
5
a
x
10
b
15
Cumulative Distribution function, F(x)
 0
xa

F  x   P  X  x  
b  a
 1
F  x
xa
a xb
xb
1
0.5
0
0
a
5
b
10
x
15
The Normal distribution
Definition: A random variable , X, is said to have
a Normal distribution with mean m and standard
deviation s if X is a continuous random variable
with probability density function f(x):

1
f  x 
e
2s
 x  m 2
2s 2
Graph: the Normal Distribution
(mean m, standard deviation s)
s
m
Note:

1
f  x 
e
2s

1
f  x 
e
2s
 x  m 2
2s 2
 x  m 2
2s 2
 12  x  m   0
 s

if x  m  0 or x  m
Thus the point m is an extremum point of f(x). (In this
case a maximum)
1
d 
f   x   
e
2s 3 dx

1
2s
5
e

 x  m 2
2s 2
 x  m 2
2s 2
x  m
x  m
2

1
2s
3
e

 x  m 2
2s 2
2


x

m
 2


1
2s

e

1

0


2
2s 3
 s

2
 x  m   1 or x  m  1 i.e. x  m  s
s2
s
 x  m 2
if
Thus the points m – s, m + s are points of inflection of
f(x)



1
e
2s
 f  x  dx  
Also

Proof:



To evaluate


1
e
2s
Make the substitution z 
dz 
1
s
 x  m 2
2s 2
dx  1
 x  m 2
2s 2
dx
xm
s
dx
When x  , z   and when x  , z  .




1
e
2s
 x  m 2
2s 2

dx 


1
e
2
z2

2
1
dz 
2

e

z2

2
dz

e
Consider evaluating
z2

2
dz  c

Note:

c   e

 

2
 

 e
 
z2

2


dz   e

 
2
z
2
e


z2

2
 
2
u
2
   z2  
dz     e dz    e


  
  

dudz 
 e
 z 2 u 2 


2



dudz
 
Make the change to polar coordinates (R, q)
z = R sin(q) and u = R cos(q)
u2

2

du 


z
Hence R  z  u and tan q  
u
1  z 
2
2
or R  z  u and q  tan  
u
2
2
2
Using
 
2 
 
0 0
  f  z, u dzdu    f  R sin q  , R cos q  RdRdq
 
c 
2
 e
 
2 


 z 2 u 2 


2


e
0 0
 R2 


2


dudz 
RdRdq 
2

0
 e

2
 R2

2
 dq  1dq  2
0
 0

and
c
e
z2

2
dz  2

or

1


1
e
2
z2

2

dz 


1
e
2s
2
xm 


2s 2
dz
The Exponential distribution
Consider a continuous random variable, X with the
following properties:
1.
2.
P[X ≥ 0] = 1, and
P[X ≥ a + b] = P[X ≥ a] P[X ≥ b] for
all a > 0, b > 0.
These two properties are reasonable to assume if X =
lifetime of an object that doesn’t age.
The second property implies:
P  X  a  b
P  X  a
 P  X  a  b X  a   P  X  b 
The property:
P  X  a  b X  a   P  X  b 
models the non-aging property
i.e. Given the object has lived to age a, the probability
that is lives a further b units is the same as if it was born
at age a.
Let F(x) = P[X ≤ x] and G(x) = P[X ≥ x] .
Since X is a continuous RV then G(x) = 1 – F(x)
(P[X ≥ x] = 0 for all x.)
The two properties can be written in terms of G(x):
1.
2.
G(0) = 1, and
G(a + b) = G(a) G(b) for all a > 0, b > 0.
We can show that the only continuous function, G(x),
that satisfies 1. and 2. is a exponential function
From property 2 we can conclude
G  2a   G  a  a   G  a  G  a   G  a 
2
Using induction
G  na   G  a 
 a  G a
G  a   G  a 
Hence putting a = 1.
G  n   G 1
n
Also putting a = 1/n.
G 1  G   or G    G 1
Finally putting a = 1/m.
n
n
1
n
m 
m

n
1
G  m   G  m   G 1
 G 1


1
n
n
1
n
1
n
n
Since G(x) is continuous
G  x   G 1
x
for all x ≥ 0.
Now 0  G 1  1 Also G  0  1 and G    0.
If G(1) = 0 then G(x) = 0 for all x > 0 and G(0) = 0 if G
is continuous. (a contradiction)
If G(1) = 1 then G(x) = 1 for all x > 0 and G() = 1 if G
is continuous. (a contradiction)
Thus G(1) ≠ 0, 1 and 0 < G(1) < 1
Let l = - ln(G(1)) then G(1) = e-l
Thus G  x   G 1   e   e  l x
x
l
x
 0
Finally F  x   1  G  x   
l x
1

e

x0
x0
To find the density of X we use:
lel x
f  x  F x  
 0
x0
x0
A continuous random variable with this density
function is said to have the exponential distribution
with parameter l.
Graphs of f(x) and F(x)
1
f(x)
0.5
0
-2
0
2
4
6
1
F(x)
0.5
0
-2
0
2
4
6
Another derivation of the Exponential distribution
Consider a continuous random variable, X with the
following properties:
1.
2.
P[X ≥ 0] = 1, and
P[x ≤ X ≤ x + dx|X ≥ x] = ldx
for all x > 0 and small dx.
These two properties are reasonable to assume if X =
lifetime of an object that doesn’t age.
The second property implies that if the object has lived
up to time x, the chance that it dies in the small interval
x to x + dx depends only on the length of that interval,
dx, and not on its age x.
Determination of the distribution of X
Let F (x ) = P[X ≤ x] = the cumulative distribution
function of the random variable, X .
Then P[X ≥ 0] = 1 implies that F(0) = 0.
Also P[x ≤ X ≤ x + dx|X ≥ x] = ldx implies
P  x  X  x  dx X  x  

or
F  x  dx   F  x 
dx

P  x  X  x  dx 
P  X  x
F  x  dx   F  x 
1 F  x
dF  x 
dx
 l dx
 l 1  F  x  
We can now solve the differential equation
dF
for the unknown F.
 l 1  F 
dx
or
1
dF  l dx
1 F
1
 1  F dF  l  dx
and  ln 1  F   l x  c
ln 1  F   l x  c
and 1  F  el xc or F  F  x   1  el xc
Now using the fact that F(0) = 0.
F  0  1  ec  0 implies ec  1 and c  0
Thus F  x   1  el x and f  x   F   x   lel x
This shows that X has an exponential distribution
with parameter l.
The Weibull distribution
A model for the lifetime of objects
that do age.
Recall the properties of continuous random variable, X,
that lead to the Exponential distribution. namely
1.
2.
P[X ≥ 0] = 1, and
P[x ≤ X ≤ x + dx|X ≥ x] = ldx
for all x > 0 and small dx.
Suppose that the second property is replaced by:
2.
P[x ≤ X ≤ x + dx|X ≥ x] = lx)dx = axb – 1dx
for all x > 0 and small dx.
The second property now implies that object does age.
If it has lived up to time x, the chance that it dies in the
small interval x to x + dx depends both on the length of
that interval, dx, and its age x.
Derivation of the Weibulll distribution
A continuous random variable, X satisfies the following
properties:
1.
2.
P[X ≥ 0] = 1, and
P[x ≤ X ≤ x + dx|X ≥ x] = axb -1dx
for all x > 0 and small dx.
Let F (x ) = P[X ≤ x] = the cumulative distribution
function of the random variable, X .
Then P[X ≥ 0] = 1 implies that F(0) = 0.
Also P[x ≤ X ≤ x + dx|X ≥ x] = axb -1dx implies
P  x  X  x  dx X  x  

or
F  x  dx   F  x 
dx

P  x  X  x  dx 
P  X  x
F  x  dx   F  x 
1 F  x
dF  x 
dx
 a x b 1dx
 a x b 1 1  F  x  
We can now solve the differential equation
dF
 a x b 1 1  F  for the unknown F.
dx
1
dF  a x b 1dx
1 F
1
or 
dF  a  x b 1dx
1 F
xb
and  ln 1  F   a
c
b
a b
ln 1  F    x  c
b
1 F  e
a
b
 x b c
or F  x   1  e
a
b
 xb c
again F(0) = 0 implies c = 0.
Thus
and
F  x  1 e
a
b
 xb
f  x   F   x   a x b 1e
a
b
 xb
x0
The distribution of X is called the Weibull
distribution with parameters a and b.
The Weibull density, f(x)
0.7
(a = 0.9, b = 2)
0.6
(a = 0.7, b = 2)
0.5
0.4
(a = 0.5, b = 2)
0.3
0.2
0.1
0
0
1
2
3
4
5
The Gamma distribution
An important family of distibutions
The Gamma Function, G(x)
An important function in mathematics
The Gamma function is defined for x ≥ 0 by

G  x    u x 1eu du
0
u x 1e  u
G  x
u
Properties of Gamma Function, G(x)
1.
G(1) = 1




G 1   e du  e   e  e0  1
0
u
u
0
2.
G(x + 1) = xG(x)

G  x  1   u e du   u e du
0
x u
x u

0
We will use integration by parts to evaluate  u x e  u du
x u
u
 e du   wdv  wv   vdw
where w  u x and dv  eu du, hence dw  xu x1dx, v  eu
Thus
x u
x u
x 1  u
u
e
du


u
e

x
u

 e du

and
x u 
x u
u
 e du  u e
0

 x  u x 1eu du
0
0
or G  x  1  0  xG  x 
3.
G(n) = (n - 1)! for any positive integer n
using G  x  1  xG  x  and G 1  1
G  2  1G 1  1  1!
G  3  2G  2  2  2!
G  4  3G  3  3  2  3!
4.
G  12   

G  x    u e du thus G 
0
x 1  u

1
2
  u
 12  u
e du
0
Recall the density for the normal distribution
 x  m 2
 2
1
f  x 
e 2s
2s
If m   and s  1 then
1  x22
f  x 
e
2



2
2
x
x
1 2
1 2
2  x22
Now 1= 
e dx  2
e dx  
e dx
2
2


0
0

e
Thus

2
 x2
dx =
Make the substitution u 
2
0
x2
2
.
 12
1
u
Hence x  2u and du  xdx or dx  du 
du.
x
2
Also when x  0,  then u  0, 
1
2

2

2
 x2

=  e dx   e
0
and G  12  =  .
0
u
u
 12
1
dx 
G  12 
2
2
5.
G n 
1
2
2n  !

  2n 
n !2
Using G  x  1  xG  x  .
G  32   12 G  12  = 12  .
G  52   32 G  32  = 32 12  .
G  72   52 G  52  = 52
3 1
2 2
.
G  2 n21   G  n  12  = 2 n21

5 3 1
2 2 2
 2n  ! 
n
n
2 n !2

2n  !



2n
n!2
Graph: The Gamma Function
25
G(x)
20
15
10
5
0
0
1
2
3
4
x
5
The Gamma distribution
Let the continuous random variable X have
density function:
 l a a 1  l x
x e

f  x    G a 

0

x0
x0
Then X is said to have a Gamma distribution
with parameters a and l.
Graph: The gamma distribution
0.4
(a = 2, l = 0.9)
0.3
(a = 2, l = 0.6)
(a = 3, l = 0.6)
0.2
0.1
0
0
2
4
6
8
10
Comments
1. The set of gamma distributions is a family of
distributions (parameterized by a and l).
2. Contained within this family are other distributions
a. The Exponential distribution – in this case a = 1, the
gamma distribution becomes the exponential distribution
with parameter l. The exponential distribution arises if
we are measuring the lifetime, X, of an object that does
not age. It is also used a distribution for waiting times
between events occurring uniformly in time.
b. The Chi-square distribution – in the case a = n/2 and
l = ½, the gamma distribution becomes the chi- square
(c2) distribution with n degrees of freedom. Later we
will see that sum of squares of independent standard
normal variates have a chi-square distribution, degrees
of freedom = the number of independent terms in the
sum of squares.
The Exponential distribution
lel x
f  x  
 0
x0
x0
The Chi-square (c2) distribution with n d.f.
  1  2 n 1  1 x
 2 x2 e 2
f  x    G  n2 

0

n
n 2  x
 1
2
2
x
e
n
 2 n
 2 G 2 

0

x0
x0
x0
x0
Graph: The c2 distribution
(n = 4)
0.2
(n = 5)
(n = 6)
0.1
0
0
4
8
12
16
Summary
Important continuous distributions
Uniform distribution from a to b
 1

f  x  b  a
 0
xa
 0
xa

F  x   P  X  x  
b  a
 1
a xb
otherwise
a xb
xb
0.4
f  x
F  x
0.3
0.2


1 

ba 

0.1
0
0
1
0.5
5
10
a
b
x
0
15
0
a5
b
10
x
15
Normal distribution
mean m and standard deviation s

1
f  x 
e
2s
 x  m 2
2s 2
s
m
the exponential distribution with parameter l
lel x
  x = 
f  x   Ff(x)
 0
x0
x0
1
f(x)
0.5
0
-2
0
2
4
6
The Weibull distribution - parameters a and b.
and
b 1
 x=
f  x   Ff(x)

a
x
e

0.7
a
b
 xb
x0
(a = 0.9, b = 2)
0.6
(a = 0.7, b = 2)
0.5
0.4
(a = 0.5, b = 2)
0.3
0.2
0.1
0
0
1
2
3
4
5
The gamma distribution - parameters a and l
 l a a 1  l x
x e

f  x    G a 

0

0.4
x0
x0
(a = 2, l = 0.9)
0.3
(a = 2, l = 0.6)
(a = 3, l = 0.6)
0.2
0.1
0
0
2
4
6
8
10
The c2 distribution with n degrees of freedom
  1  2 n 1  1 x
 2 x2 e 2
f  x    G  n2 

0

n
x0
x0
(n = 4)
0.2
(n = 5)
(n = 6)
0.1
0
0
4
8
12
16