Download Random Variables

Discrete distributions
The Bernoulli distribution
1
0.8
q  1  p
p  x   P  X  x  
p

x0
x 1
0.6
0.4
0.2
0
0
1 Bernoulli trial = S
X 
0 Bernoulli trial = F
1
The Binomial distribution
 n  x n x
p  x   P  X  x    p q
x  0,1, 2,
 x
,n
0.3000
p(x)
0.2500
0.2000
0.1500
0.1000
0.0500
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
x
X = the number of successes in n repetitions of
a Bernoulli trial
p = the probability of success
The Poisson distribution
x
p  x 
x!
e 
x  0,1, 2,3, 4,
Events are occurring randomly and uniformly in time.
X = the number of events occuring in a fixed period of
time.
0.12
0.10
0.08
0.06
0.04
0.02
0
2
4
6
8
10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40
The Geometric distribution
the Bernoulli trials are repeated independently the first success
occurs (,k = 1) and X = the trial on which the 1st success
occurred.
P[X = x] = p(x) = p(1 – p)x – 1 = pqx – 1
The Negative Binomial distribution
the Bernoulli trials are repeated independently until a fixed
number, k, of successes has occurred and X = the trial on which
the kth success occurred.
 x  1 k x  k
p  x   P  X  x  
p q
 k  1
x  k , k  1, k  2,
Geometric ≡ Negative Binomial with k = 1
The Hypergeometric distribution
Suppose we have a population containing N objects.
The population are partitioned into two groups.
• a = the number of elements in group A
• b = the number of elements in the other group (group B).
Note N = a + b.
• n elements are selected from the population at random.
• X = the elements from group A. (n – X will be the number of
elements from group B.)
 a  b 
 

x  n  x 

p  x   P  X  x 
N
 
n
Example: Hyper-geometric distribution
Suppose that N = 10 automobiles have just come off the
production line. Also assume that a = 3 are defective
(have serious defects). Thus b = 7 are defect-free.
A sample of n = 4 are selected and tested to see if they
are defective. Let X = the number in the sample that are
defective. Find the probability function of X.
From the above discussion X will have a hypergeometric distribution i.e.
 a  b   3  7 
 
  

x
n

x
x
4

x
   
 x  0,1, 2,3
p  x   P  X  x    
N
10 
 
 
n
 
4
Table and Graph of p(x)
x
p (x )
0
0.1667
1
0.5000
2
0.3000
3
0.0333
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0
1
2
3
Sampling with and without replacement
Suppose we have a population containing N objects.
Suppose the elements of the population are partitioned
into two groups. Let a = the number of elements in group
A and let b = the number of elements in the other group
(group B). Note N = a + b.
Now suppose that n elements are selected from the
population at random. Let X denote the elements from
group A. (n – X will be the number of elements from
group B.)
Find the probability distribution of X.
1. If the sampling was done with replacement.
2. If the sampling was done without replacement
Solution:
1. If the sampling was done with replacement.
Then the distribution of X is the Binomial distn.
with
a
b
p
and q  1  p 
N
N
x
n x
n
  a   b 
i.e. pBinom  x   P  X  x       
 x N   N 
2. If the sampling was done without replacement.
Then the distribution of X is the hyper-geometric distn.
 a  b 
 

x  n  x 

i.e. pHyper  x   P  X  x  
N
 
n
Note:
 a  b   N 
  
pHyper  x   x 
n

x
 n
 
pBinom  x   n   a  x  b  n  x
    
 x N   N 
a!

x ! a  x !

a  a  1
 1
1  
a

x ! n  x ! ( N  n)!n ! N n
b!
N!
a xb n  x
 n  x ! b  n  x ! n !
 a  x  1 b  b  1  b  n  x  1
ax
bn x
Nn
N  N  1 ( N  n  1)
 x  1  1   n  x  1 
1 
1   1 

a
b
b


 
  1 as N , a, b  
1   n 1 

1   1 

N
N

 

Thus
 a  b   N 
pHyper  x    
  
 x  n  x   n 
 n a   b 
pBinom  x        
 x N   N 
x
n x
for large values of N, a and b
Thus for large values of N, a and b sampling with
replacement is equivalent to sampling without
replacement.
Continuous Distributions
Continuous random variables
For a continuous random variable X the probability
distribution is described by the probability density
function f(x), which has the following properties :
1.
f(x) ≥ 0

2.
 f  x  dx  1.

3.
b
P  a  X  b   f  x  dx.
a
Graph: Continuous Random Variable
probability density function, f(x)

 f  x  dx  1.

b
P  a  X  b   f  x  dx.
a
The Uniform distribution from a to b
Definition: A random variable , X, is said to have
a Uniform distribution from a to b if X is a
continuous random variable with probability
density function f(x):
 1

f  x  b  a
 0
a xb
otherwise
Graph: the Uniform Distribution
(from a to b)
0.4
f  x
0.3
0.2


1 

ba 

0.1
0
0
5
10
a
b
x
15
The Cumulative Distribution function, F(x)
(Uniform Distribution from a to b)
xa
 0
xa

F  x   P  X  x  
b  a
 1
0.4
f  x
0.3
a xb
xb
0.2
F  x
0.1
0
0
5
a
x
10
b
15
Cumulative Distribution function, F(x)
 0
xa

F  x   P  X  x  
b  a
 1
F  x
xa
a xb
xb
1
0.5
0
0
5
10
15
a
b
x
The Normal distribution
Definition: A random variable , X, is said to have
a Normal distribution with mean m and standard
deviation s if X is a continuous random variable
with probability density function f(x):

1
f  x 
e
2s
 x  m 2
2s 2
Graph: the Normal Distribution
(mean m, standard deviation s)
s
m
Note:

1
f  x 
e
2s

1
f  x 
e
2s
 x  m 2
2s 2
 x  m 2
2s 2
 12  x  m   0
 s

if x  m  0 or x  m
Thus the point m is an extremum point of f(x). (In this
case a maximum)
1
d 
f   x   
e
2s 3 dx

1
2s
5
e

 x  m 2
2s 2
 x  m 2
2s 2
x  m
x  m
2

1
2s
3
e

 x  m 2
2s 2
2


x

m
 2


1
2s

e

1

0


2
2s 3
 s

2
 x  m   1 or x  m  1 i.e. x  m  s
s2
s
 x  m 2
if
Thus the points m – s, m + s are points of inflection of
f(x)



1
e
2s
 f  x  dx  
Also

Proof:



To evaluate


1
e
2s
Make the substitution z 
dz 
1
s
 x  m 2
2s 2
dx  1
 x  m 2
2s 2
dx
xm
s
dx
When x  , z   and when x  , z  .




1
e
2s
 x  m 2
2s 2

dx 


1
e
2
z2

2
1
dz 
2

e

z2

2
dz

e
Consider evaluating
z2

2
dz  c

Note:

c   e

 

2
 

 e
 
z2

2


dz   e

 
2
z
2
e


z2

2
 
2
u
2
   z2  
dz     e dz    e


  
  

dudz 
 e
 z 2 u 2 


2



dudz
 
Make the change to polar coordinates (R, q)
z = R sin(q) and u = R cos(q)
u2

2

du 


z
Hence R  z  u and tan q  
u
1  z 
2
2
or R  z  u and q  tan  
u
2
2
2
Using
 
2 
 
0 0
  f  z, u dzdu    f  R sin q  , R cos q  RdRdq
 
c 
2
 e
 
2 


 z 2 u 2 


2


e
0 0
 R2 


2


dudz 
RdRdq 
2

0
 e

2
 R2

2
 dq  1dq  2
0
 0

and
c
e
z2

2
dz  2

or

1


1
e
2
z2

2

dz 


1
e
2s
2
xm 


2s 2
dz
The Exponential distribution
Consider a continuous random variable, X with the
following properties:
1.
2.
P[X ≥ 0] = 1, and
P[X ≥ a + b] = P[X ≥ a] P[X ≥ b] for
all a > 0, b > 0.
These two properties are reasonable to assume if X =
lifetime of an object that doesn’t age.
The second property implies:
P  X  a  b
P  X  a
 P  X  a  b X  a   P  X  b 
The property:
P  X  a  b X  a   P  X  b 
models the non-aging property
i.e. Given the object has lived to age a, the probability
that is lives a further b units is the same as if it was born
at age a.
Let F(x) = P[X ≤ x] and G(x) = P[X ≥ x] .
Since X is a continuous RV then G(x) = 1 – F(x)
(P[X ≥ x] = 0 for all x.)
The two properties can be written in terms of G(x):
1.
2.
G(0) = 1, and
G(a + b) = G(a) G(b) for all a > 0, b > 0.
We can show that the only continuous function, G(x),
that satisfies 1. and 2. is a exponential function
From property 2 we can conclude
G  2a   G  a  a   G  a  G  a   G  a 
2
Using induction
G  na   G  a 
 a  G a
G  a   G  a 
Hence putting a = 1.
G  n   G 1
n
Also putting a = 1/n.
G 1  G   or G    G 1
Finally putting a = 1/m.
n
n
1
n
m 
m

n
1
G  m   G  m   G 1
 G 1


1
n
n
1
n
1
n
n
Since G(x) is continuous
G  x   G 1
x
for all x ≥ 0.
Now 0  G 1  1 Also G  0  1 and G    0.
If G(1) = 0 then G(x) = 0 for all x > 0 and G(0) = 0 if G
is continuous. (a contradiction)
If G(1) = 1 then G(x) = 1 for all x > 0 and G() = 1 if G
is continuous. (a contradiction)
Thus G(1) ≠ 0, 1 and 0 < G(1) < 1
Let  = - ln(G(1)) then G(1) = e-
Thus G  x   G 1   e   e   x
x

x
 0
Finally F  x   1  G  x   
 x
1

e

x0
x0
To find the density of X we use:
e x
f  x  F x  
 0
x0
x0
A continuous random variable with this density
function is said to have the exponential distribution
with parameter .
Graphs of f(x) and F(x)
1
f(x)
0.5
0
-2
0
2
4
6
1
F(x)
0.5
0
-2
0
2
4
6
Another derivation of the Exponential distribution
Consider a continuous random variable, X with the
following properties:
1.
2.
P[X ≥ 0] = 1, and
P[x ≤ X ≤ x + dx|X ≥ x] = dx
for all x > 0 and small dx.
These two properties are reasonable to assume if X =
lifetime of an object that doesn’t age.
The second property implies that if the object has lived
up to time x, the chance that it dies in the small interval
x to x + dx depends only on the length of that interval,
dx, and not on its age x.
Determination of the distribution of X
Let F (x ) = P[X ≤ x] = the cumulative distribution
function of the random variable, X .
Then P[X ≥ 0] = 1 implies that F(0) = 0.
Also P[x ≤ X ≤ x + dx|X ≥ x] = dx implies
P  x  X  x  dx X  x  

or
F  x  dx   F  x 
dx

P  x  X  x  dx 
P  X  x
F  x  dx   F  x 
1 F  x
dF  x 
dx
  dx
  1  F  x  
We can now solve the differential equation
dF
for the unknown F.
  1  F 
dx
or
1
dF   dx
1 F
1
 1  F dF    dx
and  ln 1  F    x  c
ln 1  F    x  c
and 1  F  e xc or F  F  x   1  e xc
Now using the fact that F(0) = 0.
F  0  1  ec  0 implies ec  1 and c  0
Thus F  x   1  e x and f  x   F   x   e x
This shows that X has an exponential distribution
with parameter .