Download Introduction to Probability Theory

Random Variables
Discrete Random Variables
For a discrete random variable X the probability
distribution is described by the probability function
p(x) = P[X = x], which has the following properties:
1.
0  p  x  1

2.
 p  x   p  x   1
x
3.
i 1
P  a  x  b 
i
 p  x
a  x b
Continuous random variables
For a continuous random variable X the probability
distribution is described by the probability density
function f(x), which has the following properties :
1.
f(x) ≥ 0

2.
 f  x  dx  1.

3.
b
P  a  X  b   f  x  dx.
a
The distribution function F(x)
This is defined for any random variable, X.
F(x) = P[X ≤ x]
Properties
1.
2.
3.
F(-∞) = 0 and F(∞) = 1.
F(x) is non-decreasing (i. e. if x1 < x2 then
F(x1) ≤ F(x2) )
F(b) – F(a) = P[a < X ≤ b].
4.
p(x) = P[X = x] =F(x) – F(x-)
Here
F  x    lim F  u 
ux
5. If p(x) = 0 for all x (i.e. X is continuous)
then F(x) is continuous.
6.
For Discrete Random Variables
F  x   P  X  x   p u 
u x
F(x) is a non-decreasing step function with
F    0 and F     1
p  x   F  x   F  x    jump in F  x  at x.
1.2
F(x)
1
0.8
0.6
0.4
p(x)
0.2
0
-1
0
1
2
3
4
7.
For Continuous Random Variables
Variables
x
F  x   P  X  x    f  u  du

F(x) is a non-decreasing continuous function with
F    0 and F     1
f  x  F   x.
f(x) slope
F(x)
1
0
-1
0
1
x
2
To find the probability density function, f(x), one first
finds F(x) then
f  x  F   x.
Some Important Discrete
distributions
The Bernoulli distribution
Suppose that we have a experiment that has two
outcomes
1. Success (S)
2. Failure (F)
These terms are used in reliability testing.
Suppose that p is the probability of success (S) and
q = 1 – p is the probability of failure (F)
This experiment is sometimes called a Bernoulli Trial
Let
0 if the outcome is F
X 
1 if the outcome is S
q
Then p  x   P  X  x   
p
x0
x 1
The probability distribution with probability function
q x  0
p  x   P  X  x  
p x 1
is called the Bernoulli distribution
1
0.8
0.6
p
q = 1- p
0.4
0.2
0
0
1
The Binomial distribution
We observe a Bernoulli trial (S,F) n times.
Let X denote the number of successes in the n trials.
Then X has a binomial distribution, i. e.
 n  x n x
p  x   P  X  x    p q
x  0,1, 2,
 x
where
1. p = the probability of success (S), and
2. q = 1 – p = the probability of failure (F)
,n
Example
A coin is tossed n= 7 times.
Let X denote the number of heads (H) in the n = 7
trials.
Then X has a binomial distribution, with p = ½ and
n = 7.
Thus
 n  x n x
p  x   P  X  x    p q
x  0,1, 2,
 x
 7  1 x 1 7 x
   2   2 
x  0,1, 2, , 7
 x
7 1 7
   2 
 x
x  0,1, 2,
,7
,n
x
0
1/
p(x)
p(x)
128
1
7/
128
2
21/
128
3
35/
128
4
35/
5
128
21/
6
7/
128
7
1/
128
0.3
0.25
0.2
0.15
0.1
0.05
0
0
1
2
3
4
x
5
6
7
128
Example
If a surgeon performs “eye surgery” the chance of “success”
is 85%. Suppose that the surgery is perfomed n = 20 times
Let X denote the number of successful surgeries in the n =
20 trials.
Then X has a binomial distribution, with p = 0.85 and n =
20.
Thus
 n  x n x
p  x   P  X  x    p q
x  0,1, 2, , n
 x
 20 
x
7 x
   .85  .15
x  0,1, 2, , 20
 x
x
p (x )
x
p (x )
x
p (x )
x
p (x )
0
0.0000
6
0.0000
12
0.0046
18
0.2293
1
0.0000
7
0.0000
13
0.0160
19
0.1368
2
0.0000
8
0.0000
14
0.0454
20
0.0388
3
0.0000
9
0.0000
15
0.1028
4
0.0000
10
0.0002
16
0.1821
5
0.0000
11
0.0011
17
0.2428
0.3000
0.2500
p(x)
0.2000
0.1500
0.1000
0.0500
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
x
The probability that at least sixteen operations are
successful
= P[X ≥ 16]
= p(16) + p(17) + p(18) + p(19) + p(20)
= 0.1821 + 0.2428 + 0.2293 + 0.1368 + 0.0388
= 0.8298
Other discrete distributions
•
•
•
•
Poisson distribution
Geometric distribution
Negative Binomial distribution
Hypergeometric distribution
The Poisson distribution
• Suppose events are occurring randomly and
uniformly in time.
• Let X be the number of events occuring in a
fixed period of time. Then X will have a
Poisson distribution with parameter l.
p  x 
lx
x!
e
l
x  0,1, 2,3, 4,
Some properties of the probability function for
the Poisson distribution with parameter l.

x 0
x 0
 p  x  
1.


l
x
 x! e
x 0
l
l
x
x!
e l  1
2
3
4

l l l
l
 e 1  l 
 

2! 3! 4!

 e  l  el   1
2
3
4
u
u
u
u
using e  1  u    
2! 3! 4!



n x
n x
2. If pBin  x p, n     p 1  p 
 x
is the probability function for the Binomial
distribution with parameters n and p, and we
allow n → ∞ and p → 0 such that np = a
constant (= l say) then
l x l
lim pBin  x p, n   pPoisson  x l  
e
n  , p 0
x!
n x
n x
Proof: pBin  x p, n     p 1  p 
 x
l
Suppose np  l or p 
n
x
n!
l
pBin  x p, n   pBin  x l , n  
 
x ! n  x !  n 
l
x
 l
1  
 n
n x
n!
 l  l

1   1  
x
x ! n  n  x !  n   n 
x
n
x
n
n

1
n

x

1




l
 l  l

1   1  
x!
nn n
 n  n
x
n
x
l  1   x  1  l   l 
 1 1    1 
1   1  
x!  n  
n  n   n 
x
n
Now
lim pBin  x l , n 
n 

l
 1 

lim 1  
x ! n 
 n 
x
l
 l

lim 1  
x ! n  n 
x

 x  1  l   l  
1 
1   1   
n  n   n  


x
n
n
n
Now using the classic limit
lim pBin  x l , n  
n 
l
 u
lim  1    eu
n 
 n
l
n
lx

lim 1   
e  l  pPoisson  x l 
x ! n  n 
x!
x
Graphical Illustration
Suppose a time interval is divided into n equal parts and
that one event may or may not occur in each subinterval.
n subintervals
time interval
- Event occurs
- Event does not occur
X = # of events is Bin(n,p)
As n→∞ , events can occur over the continuous time
interval.
X = # of events is Poisson(l)
Example
The number of Hurricanes over a period of a
year in the Caribbean is known to have a Poisson
distribution with l = 13.1
Determine the probability function of X.
Compute the probability that X is at most 8.
Compute the probability that X is at least 10.
Given that at least 10 hurricanes occur, what is
the probability that X is at most 15?
Solution
• X will have a Poisson distribution with
parameter l = 13.1, i.e.
p  x 
l
x
x!
e
l
x
13.1 13.1

e
x!
x  0,1, 2,3, 4,
x  0,1, 2,3, 4,
Table of p(x)
x
0
1
2
3
4
5
6
7
8
9
p (x )
0.000002
0.000027
0.000175
0.000766
0.002510
0.006575
0.014356
0.026866
0.043994
0.064036
x
10
11
12
13
14
15
16
17
18
19
p (x )
0.083887
0.099901
0.109059
0.109898
0.102833
0.089807
0.073530
0.056661
0.041237
0.028432
P at most 8  P  X  8
 p  0  p 1 
 p 8  .09527
P at least 10  P  X  10  1  P  X  9
 1   p  0   p 1 
 p  9    .8400
P at most 15 at least 10  P  X  15 X  10


P  X  15   X  10
P  X  10
p 10   p 11 
.8400

 p 15
P 10  X  15
P  X  10
 0.708
The Geometric distribution
Suppose a Bernoulli trial (S,F) is repeated until a
success occurs.
Let X = the trial on which the first success (S) occurs.
Find the probability distribution of X.
Note: the possible values of X are {1, 2, 3, 4, 5, … }
The sample space for the experiment (repeating a
Bernoulli trial until a success occurs is:
S = {S, FS, FFS, FFFS, FFFFS, … , FFF…FFFS, …}
(x – 1) F’s
p(x) =P[X = x] = P[{FFF…FFFS}] = (1 – p)x – 1p
Thus the probability function of X is:
P[X = x] = p(x) = p(1 – p)x – 1 = pqx – 1
A random variable X that has this distribution is said to
have the Geometric distribution.
Reason p(1) = p, p(2) = pq, p(3) = pq2 , p(4) = pq3 , …
forms a geometric series

 p  x   p 1 +p  2  +p 3 +
x 1
 p  pq  pq  pq 
2
3
p
p

 1
1- q p
The Negative Binomial distribution
Suppose a Bernoulli trial (S,F) is repeated until k
successes occur.
Let X = the trial on which the kth success (S) occurs.
Find the probability distribution of X.
Note: the possible values of X are
{k, k + 1, k + 2, k + 3, 4, 5, … }
The sample space for the experiment (repeating a
Bernoulli trial until k successes occurs) consists of
sequences of S’s and F’s having the following properties:
1. each sequence will contain k S’s
2. The last outcome in the sequence will be an S.
A sequence of length x
containing exactly k S’s
SFSFSFFFFS FFFSF … FFFFFFS
The # of S’s in
the first x – 1
trials is k – 1.
The last
outcome
is an S
 x  1 k x  k
p  x   P  X  x  
p q
 k  1
x  k , k  1, k  2,
The # of ways of
choosing from the
first x – 1 trials, the
positions for the
first k – 1 S’s.
The probability
of a sequence
containing k S’s
and x – k F’s.
The Hypergeometric distribution
Suppose we have a population containing N objects.
Suppose the elements of the population are partitioned
into two groups. Let a = the number of elements in group
A and let b = the number of elements in the other group
(group B). Note N = a + b.
Now suppose that n elements are selected from the
population at random. Let X denote the elements from
group A. (n – X will be the number of elements from
group B.)
Find the probability distribution of X.\
Population
GroupB (b elements)
Group A (a elements)
x
sample (n elements)
N-x
Thus the probability function of X is:
The number of ways
x elements can be
chosen Group A .
 a  b 
 

x
n

x

p  x   P  X  x    
N
 
n
The number of ways
n - x elements can be
chosen Group B .
The total number of
ways n elements can be
chosen from N = a + b
elements
A random variable X that has this distribution is said to have the
Hypergeometric distribution.
The possible values of X are integer values that range from
max(0,n – b) to min(n,a)