Download Week2

Conditional Probability
• Idea – have performed a chance experiment but don’t know the outcome
(ω), but have some partial information (event A) about ω.
Question: given this partial information what’s the probability that the
outcome is in some event B?
• Example:
Toss a coin 3 times. We are interested in event B that there are 2 or more
heads. The sample space has 8 equally likely outcomes.
  HHH , HHT , HTH , THH , HTT , THT , TTH , TTT 
The probability of the event B is …
Suppose we know that the first coin came up H. Let A be the event the first
outcome is H. Then A  HHH , HHT , HTH , HTT  and A  B  HHH , HHT , HTH 
The conditional probability of B given A is 3  3 8  P A  B 
4
week 2
48
P  A
1
• Given a probability space (Ω, F, P) and events A, B  F with P(A) > 0
The conditional probability of B given the information that A has occurred is
P  B | A 
P A  B 
P  A
• Example:
We toss a die. What is the probability of observing the number 6 given that
the outcome is even?
• Does this give rise to a valid probability measure?
• Theorem
If A F and P(A) > 0 then (Ω, F, Q) is a probability space where Q : F  R
is defined by Q(B) = P(B | A).
Proof:
week 2
2
• The fact that conditional probability is a valid probability
measure allows the following:
 PB | A  1  PB | A ,
A, B  F, P(A) >0
 PB1  B2 | A  PB1 | A  PB2 | A  PB1  B2 | A
for any A, B1, B2  F, P(A) >0.
week 2
3
Multiplication rule
• For any two events A and B, P( A  B)  PB | AP A
• For any 3 events A, B and C, P A  B  C   P APB | APC | A  B
• In general,
n 1
 n 


P  Ai   P A1 P A2 | A1 P A3 | A1  A2     P An |  Ai 
i 1
 i 1 


• Example:
An urn initially contains 10 balls, 3 blue and 7 white. We draw a ball and
note its colure; then we replace it and add one more of the same colure.
We repeat this process 3 times. What is the probability that the first 2 balls
drawn are blue and the third one is white?
Solution:
week 2
4
Law of total probability
• Definition:
For a probability space (Ω, F, P), a partition of Ω is a
countable collection Bi  of events such that
Bi  F , Bi  B j   and  Bi  .
i
• Theorem:
If B1 , B2 ,... is a partition of Ω such that PBi   0  i then
P A   P A | Bi PBi 
• Proof:
for any A  F
.
i
week 2
5
Examples
1.
Calculation of PB2  for the Urn example.
2.
In a certain population 5% of the females and 8% of the males are lefthanded; 48% of the population are males. What proportion of the
population is left-handed?
Suppose 1 person from the population is chosen at random; what is the
probability that this person is left-handed?
week 2
6
Bayes’ Rule
• Let B1 , B2 ,... be a partition of Ω such that PBi   0 for all i then
PB j | A 
PA | B j PB j 
 P A | B PB 
i
for any A  F .
i
i
• Example:
A test for a disease correctly diagnoses a diseased person as having the
disease with probability 0.85. The test incorrectly diagnoses someone
without the disease as having the disease with probability 0.1 If 1% of
the people in a population have the disease, what is the probability that a
person from this population who tests positive for the disease actually
has it?
(a) 0.0085 (b) 0.0791 (c) 0.1075 (d) 0.1500 (e) 0.9000
week 2
7
Independence
• Example:
Roll a 6-sided die twice. Define the following events
A : 3 or less on first roll
B : Sum is odd.
• If occurrence of one event does not affect the probability that the other
occurs than A, B are independent.
week 2
8
• Definition
Events A and B are independent if
P A  B  P APB
• Note: Independence ≠ disjoint. Two disjoint events are independent if and
only if the probability of one of them is zero.
• Generalized to more than 2 events:
A collection of events A1 , A2 ,... An  is (mutually) independent if for any
subcollection Ai , Ai ,... Ai 

1
2
m

  
P Ai1  Ai2      Aim  P Ai1 P Ai2    P Aim 
• Note: pairwize independence does not guarantee mutual independence.
week 2
9
Example
• Roll a die twice. Define the following events;
A: 1st die odd
B: 2nd die odd
C: sum is odd.
week 2
10
Example
• Let R, S and T be independent, equally likely events with common
probability 1/3. What is PR  S  T  ?
• Solution:
week 2
11
Claim
• If A, B are independent so are A, B and A, B and A, B .
• Proof:
week 2
12
Random Variables
• Example:
We roll a fair die 6 times. Suppose we are interested in the number of 5’s in
the 6 rolls. Let X = number of 5’s. Then X could be 0, 1, 2, 3, 4, 5, 6.
X = 0 corresponds to the 56 elements of our 66 elements of Ω.
X = 1 corresponds to the elements etc.
X is an example of a random variable.
• Probability models often stated terms of random variables.
E.g. - model for the # of H’s in 10 flips of a coin.
- model for the height of a randomly chosen person.
- model for size of a queue.
week 2
13
Discrete Probability Spaces (Ω, F, P)
week 2
14
Discrete Random Variable
•
•
•
•
Definition:
A random variable X is said to be discrete if it can take only a finite or countably
infinite number of distinct values.
A discrete random variable X maps the sample space Ω onto a countable set. Define
a probability mass function (pmf) or frequency function on X such that
Where the sum is taken over all possible values of X.
Note that there is a theorem that states that there exists a probability triple and
random variable whenever we have a function p such that
Definition:
The probability distribution of a discrete random variable X is represented by a
formula, a table or a graph which provides the list of all possible values that X can
take and the pmf for each value
week 2
15
Examples of Discrete Random Variables
•
Discrete Uniform Distribution
We roll a fair die.
Let X = the # that comes up. We have that X    
This is an example of equiprobable outcomes, that is
To state the probability distribution of X we need to give its possible values and its
pmf
X is a discrete Uniform random variable. X has a uniform distribution.
week 2
16
Bernoulli Distribution
week 2
17
Binomial Distribution
• Roll a die n time and count the number of times 6 came up. Let X be the
number of 6’s in n rolls. X has image {1, 2, …, n}
The probability distribution of X is given by the following formula
• In general, if identical Bernoulli trail is repeated n times independently and
X is a random variable that count the number of success in the n trails then
the probability distribution of X is given by
Where p is the probability of success on any one experiment.
X is a Binomial random variable. X has a Binomial Distribution.
• Question: is this a valid pmf? Prove!
week 2
18
Geometric Distribution
• We roll a fair die until the first 6 comes up.
Let X = the number of rolls until we get the first 6.
Possible values of X: {1, 2, 3, …..}
The probability distribution of X is given by the following formula
• In general, if identical Bernoulli trail is repeated independently until the
first success is obtained and X is a random variable that count the number
of trials until the first success then the probability distribution of X is
given by
X is a Geometric random variable. X has a Geometric Distribution.
• Question: is this a valid pmf? Prove!
week 2
19
• In general for a Geometric distribution:
• Memory-less property of geometric random variable: for i > j
week 2
20
Negative Binomial Distribution
• We roll a fair die until the second 6 comes up. This is the waiting time for the
second 6. Let X = the number of rolls until we get two 6’s.
Possible values of X: {2, 3, 4, …..}
The probability distribution of X is given by the following formula
• Is this a valid pmf? Prove!
• In general, X is the total number of experiments when waiting for rth success in
a sequence of independent Bernoulli trails. The probability distribution of X is
given by
X has a Negative Binomial random Distribution.
week 2
21
Hypergeometric Distribution
• A hat contains 12 tickets, 7 black and 5 white. Three tickets are drawn at
random.
Let X = the # of black tickets drawn. X could be 0, 1, 2, 3.
The probability mass for each value can be calculated using combinatorics.
For example,
week 2
22
Poisson Distribution
• Model for the number of events occurring in a time (or space) interval where λ
(a parameter of the distribution) is the rate of the occurrence of the events per
one unit of time (or space).
• A Poisson random variable X = number of events per one unit of time (space).
Possible values for X: {0, 1, 2, … }
The probability distribution of X is given by
• Is this a valid pmf? Prove!
week 2
23