Download 3. Lecture Notes 3_ppt

STT 315
This lecture note is based on Chapter 3
Acknowledgement: Author is thankful to Dr. Ashok Sinha, Dr. Jennifer Kaplan
and Dr. Parthanil Roy for allowing him to use/edit some of their slides.
Some terminologies
Probability is used to infer probable feature(s) of sample
taken from a population.
• An experiment is an action that leads to one of several
possible outcomes.
• An outcome of an experiment is also called a sample
point.
• The collection of all possible outcomes (i.e. sample
points) of an experiment is known as sample space [will
be mostly denoted by S].
• An event is a subset of the sample space with which we
can attach a probability.
• We say “the event A occurred” if an outcome from A
resulted in the experiment.
2
Examples
Suppose we toss a coin and observe the resulting face at
the top.
• Experiment: tossing the coin.
• Sample points: Head (H) and Tail (T).
• Sample space: S = {H, T}.
• Events: {H}, {T}, {H,T} and Φ (the empty set).
Suppose we throw a die and observe the resulting face at
the top.
• Experiment: throwing the die.
• Sample points: 1, 2, 3, 4, 5 and 6.
• Sample space: S = {1, 2, 3, 4, 5, 6}.
• Events: any subset of S, e.g. {1}, or {2,4,6}, {3, 4, 5, 6}
etc.
3
Modeling Probability
• Suppose you have a fair/unbiased coin
(i.e., a coin with equal chance of getting a
head or a tail).
Question: If you toss a fair coin, what is the
chance of getting a head?
Answer: 50% = ½.
4
Modeling Probability
• Suppose you have a fair/unbiased die (i.e.,
a die with equal chance of getting all the
six numbers 1-6).
Question: If you toss a fair die, what is the
chance of getting a 5?
Answer: 1/6.
5
Modeling Probability
• Suppose you have a fair/unbiased die (i.e.,
a die with equal chance of getting all the
six numbers 1-6).
Question: If you toss a fair die, what is the
chance of getting an even outcome (2 or 4
or 6)?
Answer: 3/6 = ½ = 50%.
6
In General
We typically denote events (whose probabilities are
calculated) by capital letters A, B, C etc.
Probability of an event A,
P(A) =
Number of outcomes in A
Total number of possible outcomes
provided all the outcomes are equally likely.
7
Example
• Suppose you toss a fair coin twice.
What is the probability that the two
outcomes are different?
All possible outcomes = {HH, HT, TH, TT}.
Since the coin is fair, all the 4 outcomes are
equally likely.
Let A denote the event that the two
outcomes are different. Then A = {HT,
TH}.
P(A)= 2/4 = ½ .
8
Example
• Suppose you toss a fair coin twice.
What is the probability that at least one
head occurred?
All possible outcomes = {HH, HT, TH, TT}.
Since the coin is fair, all the 4 outcomes are
equally likely.
Let A denote the event that at least one had
occurred. Then A = {HH, HT, TH}.
P(A)= ¾.
9
Rules of Probability
Basic rules:
1. Probability of any event is a number between
0 and 1, i.e. for any event A, 0 ≤ 𝑃 𝐴 ≤ 1.
2. P 𝑆 = 1.
3. P 𝐴𝑐 = 𝑃 𝑛𝑜𝑡 𝐴 = 1 − 𝑃 𝐴 .
4. P 𝐴 ∪ 𝐵 = 𝑃 𝐴 + 𝑃 𝐵 − 𝑃 𝐴 ∩ 𝐵 ,
i.e., 𝑃 𝐴 𝑜𝑟 𝐵 = 𝑃 𝐴 + 𝑃 𝐵 − 𝑃 𝐴 𝑎𝑛𝑑 𝐵 .
10
Complement of a set
The complement of an event A (denoted by Ac) is
the event that A does not occur.
Ac means “not A”.
P 𝐴𝑐 = 𝑃 𝑛𝑜𝑡 𝐴 = 1 − 𝑃 𝐴 .
11
An Example
• Suppose that the probability that the traffic light
is green on a crossing is 35%. What is the
chance that the light is not green?
P(not green) = 1-P(green) = 1 - 0.35 = 0.65.
12
Intersection of sets
• “A and B” denotes the event that A and B
both occur, denoted by 𝐴 ∩ 𝐵.
• It consists of all outcomes that belong to
both A and B.
𝑨∩𝑩
13
Union of sets
“A or B” means either A occurs or B occurs or both
occur (i.e., “or” means and/or). It is denoted by 𝐴 ∪
𝐵, and is called “A union B”.
In this picture,
“𝐴 ∪ 𝐵” is the
complete
colored region.
(𝐴 ∪ 𝐵)
14
Example of 𝐴 ∩ 𝐵 and 𝐴 ∪ 𝐵
Suppose you toss a fair coin twice.
Here S = {HH, HT, TH, TT}.
Define A = second outcome is head = {HH, TH}
and B = first outcome is tail = {TH, TT}.
Then 𝐴 ∩ 𝐵 = {TH} = the first outcome is tail and the second
outcome is head,
and 𝐴 ∪ 𝐵 = {HH, TH, TT}.
15
How to find P 𝐴 ∪ 𝐵 ?
P 𝐴∪𝐵 =𝑃 𝐴 +𝑃 𝐵 −𝑃 𝐴∩𝐵 .
16
Example of A ∪ 𝐵
• Suppose I pick a student at random from
this class. Let A be the event that the
chosen student is a vegetarian and B be
the event that he/she is a junior.
• Suppose further that 6% of the students
present in this class are vegetarians and
30% are juniors and only 2% are both
junior and vegetarian.
• What is the chance that selected student is
either a vegetarian or a junior? P 𝐴 ∪ 𝐵 =?
17
Example of A ∪ 𝐵
• Suppose I pick a student at random from this
class. Let A be the event that the chosen student
is a vegetarian and B be the event that he/she is
a junior.
• Suppose further that 6% of the students present
in this class are vegetarians and 30% are juniors
and only 2% are both junior and vegetarian.
• What is the chance that selected student is either
a vegetarian or a junior?
• P(A or B) = P(A) + P(B) - P(A and B)
= 0.06 + 0.30 - 0.02 = 0.34.
18
Another Example
• A survey found out that 56% of MSU students live in
a campus dorm, 62% participate in a campus meal
program and 42% do both.
• What is the probability that a randomly selected MSU
student either lives on campus dorm or has a campus
meal plan?
Let L = {student lives on campus}, and
M = {student has a campus meal plan}.
P(either lives on campus or has a campus meal plan)
= P(L or M) = P(L) + P(M) - P(L and M)
= 0.56 + 0.62 - 0.42 = 0.76.
19
Another Example
• A survey found out that 56% of MSU students live in a
campus dorm, 62% participate in a campus meal program
and 42% do both.
• What is the probability that a randomly selected MSU
student lives on campus but does not have a campus meal
plan?
Let L = {student lives on campus} and
M = {student has a campus meal plan}.
P(a student lives on campus but does not have a campus
meal plan)
= P[L and (not M)] = ?
20
Solve Using a Picture
P[L and (not M)] = P(L) - P(L and M)
= 0.56 - 0.42 = 0.14.
21
Similarly …
P[M and (not L)] = P(M)-P(L and M) = 0.62-0.42 = 0.20.
Therefore, there is a 20% chance that a randomly selected
MSU student lives off campus but participates in a campus
meal program.
22
Another Example
• A survey found out that 56% of MSU students live in a
campus dorm, 62% participate in a campus meal program
and 42% do both.
• What is the probability that a randomly selected MSU
student neither lives on campus nor participates in a
campus meal plan?
Let L = {student lives on campus}, and
M = {student has a campus meal plan}.
P(neither lives on campus nor participates in a campus meal
plan)
= P(neither L nor M)= ?
23
Once again, use a picture 
P(neither L nor M) = P[not(L or M)] = 1 - P(L or M)
= 1 - [P(L) + P(M) - P(L and M)]
= 1 - (0.56 + 0.62 - 0.42) = 0.24.
24
Another Example
• A survey found out that 56% of MSU students live in a
campus dorm, 62% participate in a campus meal
program and 42% do both.
• What is the probability that a randomly selected MSU
student either lives on campus or participates in a
campus meal plan but not both?
Let L = {student lives on campus}, and
M = {student has a campus meal plan}.
P(either lives on campus or participates in a campus
meal plan but not both)
= P(exactly one of L and M) = ?
25
Draw Picture Again!
P(exactly one of L and M)
= P[L and (not M)] + P[M and (not L)]
= 0.14+0.20 = 0.34.
26
Disjoint Events
Two events A and B are called disjoint or mutually
exclusive if they can never occur together, i.e. A ∩
𝐵 = ∅ (empty set).
If A and B are disjoint then, 𝑃 𝐴 ∩ 𝐵 = 0.
27
Example of Disjoint Events
Suppose you toss a fair coin twice.
Here S = {HH, HT, TH, TT}.
Define A = both outcomes are head = {HH},
and B = first outcome is tail = {TH, TT}.
Then A and B are disjoint because they can
never occur together.
28
Independence
29
Independent Events
Two events A and B are called independent if
𝑃 𝐴∩𝐵 =𝑃 𝐴 𝑃 𝐵 .
Roughly speaking, this means that the
occurrence (or non-occurrence) of A has nothing
to do with the occurrence (or non-occurrence) of
B.
30
Example of Independent Events
Suppose you toss a fair coin twice.
Here S = {HH, HT, TH, TT}.
Define A = second outcome is head = {HH, TH},
and B = first outcome is tail = {TH, TT}.
Then A and B = {TH} and P(A and B)= ¼ while
P(A) = P(B) = 2/4 = ½ and therefore
P(A and B) = ¼ = ½ × ½ =P(A)P(B), which
means A and B are independent.
31
Disjoint and Independent
• Suppose A and B are two events.
• If A and B are disjoint then
𝑃 𝐴 ∩ 𝐵 = 0.
• On the other hand, A and B are
independent if
𝑃 𝐴∩𝐵 =𝑃 𝐴 𝑃 𝐵 .
32
Disjoint and Independent
• If two events A and B are disjoint then can
they be independent as well?
Ans: Only if either P(A)=0, or P(B)=0, or both.
This is because if A and B are disjoint and
independent then
0=𝑃 𝐴∩𝐵 =𝑃 𝐴 𝑃 𝐵 ,
and hence either P(A)=0, or P(B)=0, or both.
33
Disjoint and Independent: An Example
Suppose a word is chosen at random from the
sentence “Dave loves cupcakes” and then a letter
is selected at random from the chosen word.
Suppose A is the event that the word “cupcakes” is
chosen and B is the event that the letter “v” is
selected.
Then A and B are
1) independent,
2) disjoint.
3) neither.
34
Disjoint and Independent: Another Example
Suppose a fair coin is tossed twice.
Suppose A = head occurs in the first toss
and
B = head occurs in the second toss.
Then A and B are
1. independent,
2. disjoint.
3. neither.
35
Application of Independence
If it is known that two events A and B are
independent only then we can calculate P(A
and B) as follows:
𝑃 𝐴∩𝐵 =𝑃 𝐴 𝑃 𝐵 .
Sometimes the independence of A and B is
assumed implicitly although it is not explicitly
stated.
36
Example
• If the probability that one will encounter a red
light in a traffic signal is 0.61 then find the
probability that you will find the light to be red
on both Monday and Tuesday.
Here “red on Monday” and “red on Tuesday” are
implicitly assumed to be independent.
Therefore, P(red on Monday and red on Tuesday)
=P(red on Monday) × P(red on Tuesday)
=(0.61) × (0.61) = 0.3721.
37
More on Independence
More than two events can also be independent.
For example if 4 events A, B, C, D are
independent then
P(A ∩ B ∩ C ∩ D) = P(A)P(B)P(C)P(D).
Just as in the case of two events, the
independence of 4 events may not always be
stated explicitly.
38
Harder Example
• If the probability that one will encounter a red
light in a traffic signal is 0.61 then find the
probability that you will have to stop in a red
light at least once during the work week?
P(at least one red in 5 days of a week)
=1 – P(no red in 5 days of a week)
=1 – P[(no red on Monday) and (no red on Tuesday) and
(no red on Wednesday) and (no red on Thursday) and
(no red on Friday)]
=1 – (0.39) × (0.39) × (0.39) × (0.39) × (0.39)
=0.991.
39
Example
Suppose a letter is chosen at random from the sentence “Dave loves
cupcakes”.
What is the probability that the letter “v’’ will be selected?
There are total 17 letters and two of them are “v”. So
P(“v” is selected) = 2∕17 = 0.118.
40
Some Formulas
Given two events A and B.
Probability at least one of A or B occurs =P(A U B)
Probability that both A “and” B occurs =P(A “and” B)
Probability Exactly one of A or B occurs
=P(A “ and” not B) + P( not A “and” B)
Probability of Neither of A nor B =P( not A “and” not B)=1-P(AUB)
In general,
Probability of at least 1 of A and B =1-P( None of A “and” B)
41
Example
A consumer organization estimates that over a 1-year period 19% of
the cars will need to be repaired once, 12% will need repairs twice, 5%
will require three of more repairs.
Suppose two cars are chosen at random. What is the probability that:
(a) Neither will need repair ?
(b) Both will need repair ?
(c) At least one car will need repair ?
(d) Exactly 1 need repair ?
42
Conditional Probability
And
Bayes’ Rule
43
Conditional Probability
Consider the below example:
Example 1: What is probability that you get a “5” in single
throw of a die.
Example 2: If I tell you, that the number is odd, what is
probability that the number is “5” ?
Why these two probabilities are different ? Since the later, we
have more information and we condition the known information to get
the outcome. It maximizes the event outcome.
Definition:
Probability of An event “A” given event “B” is given by
P(A/B)=P(A and B) / P(B)
Another example : What is probability that tomorrow’s whether is rainy ? What is
tomorrow’s weather if if I tell you today’s weather sunny ?
44
Example
Toss a fair coin thrice.
A = at least two heads = {HHH,HHT,HTH,THH},
B = all three are heads = {HHH}.
4
8
Hence, P(A) = =
1
,
2
and P(B) =
1
.
8
What is P(B|A)? What is P(A|B)?
Here 𝐴 ∩ 𝐵 = {𝐻𝐻𝐻}.
So 𝑃 𝐵 𝐴 =
𝑃(𝐴∩𝐵)
𝑃(𝐵)
=
1
1
8
8
𝑃(𝐴∩𝐵)
𝑃(𝐴)
=
1
1
8
2
=
1
4
and 𝑃 𝐴 𝐵 =
= 1.
45
Multiplication rule
From the definition of conditional probability
that
P(A ∩ B) = P(B) P(A|B) = P(A) P(B|A).
For events A, B and C we have
P(A ∩ B ∩ C) = P(A) P(B|A) P(C| A∩B).
It can be extended similarly to larger number
of events.
46
Example
Suppose a letter is chosen at random from
the sentence “Dave loves cupcakes”.
What is the probability that the letter “v’’ will
be selected?
There are total 17 letters and two of them
are “v”. So
P(“v” is selected) = 2∕17 = 0.118.
47
Example
Suppose first a word is chosen at random from the sentence “Dave
loves cupcakes” and then a letter is selected at random from the
chosen word.
What is the probability that the letter “v’’ will be selected?
Here it depends on which word is selected first.
• P(“v” | “Dave”)
= P(“v” is selected, under the condition that “Dave” is selected first)
= ¼ = 0.25.
P(“v” | “Dave”) is the conditional probability of selecting “v” given
that “Dave” is selected first.
• P(“v” | “loves”)
= P(“v” is selected, under the condition that “loves” is selected first)
= 1∕5 = 0.2.
48
• P(“v” | “cupcakes”) = 0.
Example
Suppose first a word is chosen at random from the sentence “Dave loves
cupcakes” and then a letter is selected at random from the chosen word.
What is the probability that the letter “v’’ will be selected?
So we found that the chance of selecting “v” is NOT independent of the
words selected first.
In this case, P(“Dave” and “v” are selected)
= P(“Dave” is selected first) P(“v” | “Dave”)
= ⅓ × ¼ = 1∕12.
Similarly, P(“loves” and “v” are selected) = ⅓ × 1∕5 = 1∕15,
and P(“cupcakes” and “v” are selected) = ⅓ × 0 = 0.
So P(“v” is selected) = 1∕12 + 1∕15 + 0 = 0.15.
The computation is easily understood with the following tree diagram.
49
Example
Suppose a word is chosen at random from the sentence “Dave loves
cupcakes” and then a letter is selected at random from the chosen
word.
What is the probability that the letter “v’’ will be selected?
⅓
⅓
Dave
¼
v
⅓
loves
¾
not “v”
1/
v
5
cupcakes
4/
0
5
not “v”
v
1
not “v”
P(“v” is selected) = ⅓ × ¼ + ⅓ × 1∕5 + ⅓ × 0 = 0.15.
50
Example
Suppose first a word is chosen at random from the sentence “Dave
loves cupcakes” and then a letter is selected at random from the
chosen word.
If the selected letter is “v”, what is the conditional probability that the
word “Dave” was selected in the first place?
𝑃(Dave 𝑎𝑛𝑑 v)
𝑃 𝐷𝑎𝑣𝑒 𝑃(𝑣|𝐷𝑎𝑣𝑒)
𝑃 𝐷𝑎𝑣𝑒 𝑣 =
=
𝑃(𝑣)
0.15
1 ×1
4 = 0.556.
= 3
0.15
51
Example
The probability that an adult American man has high blood pressure and/or
cholesterol level are shown in table:
Cholesterol Level
a)
b)
c)
d)
e)
High
Ok
Blood Pressure
High
Ok
0.11
0.21
0.16
?
Find the missing probability.
What is the probability that a randomly selected man has both conditions ?
What is the probability that he has high blood pressure ?
What is the probability that a person has high blood pressure given that he
has high cholesterol ?
What is the probability that a man has high cholesterol level if it’s known
that he has high blood pressure ?
52
Example
The probability that an adult American man has high blood pressure and/or
cholesterol level are shown in table:
Cholesterol Level
a)
b)
c)
d)
e)
High
Ok
Blood Pressure
High
Ok
0.11
0.21
0.16
?
Find the missing probability.
0.52
What is the probability that a randomly selected man has both conditions ?
0.11
What is the probability that he has high blood pressure ? 0.27
What is the probability that a person has high blood pressure given that he
has high cholesterol ? .34375
What is the probability that a man has high cholesterol level if it’s known
that he has high blood pressure. 0.4074
53
Bayes’ rule
Suppose E1, …, Ek are events such that
• 𝐸1 ∪ 𝐸2 ∪ ⋯ ∪ 𝐸𝑘 = 𝑆 (exhaustive),
• 𝐸𝑖 ∩ 𝐸𝑗 = ∅ for any 𝑖 ≠ 𝑗, (mutually exclusive).
Then for any event A with 𝑃 𝐴 > 0,
𝑃(𝐸𝑗 ∩ 𝐴)
𝑃 𝐸𝑗 𝐴 =
𝑃(𝐴)
𝑃 𝐸𝑗 𝑃 𝐴|𝐸𝑗
=
.
𝑃 𝐸1 𝑃 𝐴|𝐸1 + ⋯ + 𝑃 𝐸𝑘 𝑃 𝐴|𝐸𝑘
Remark: Many a problem can be solved using tree diagram
only, without taking refuge to this complicated formula.
54
Example
Suppose first a word is chosen at random from the
sentence “Dave loves cupcakes” and then a letter is
selected at random from the chosen word.
If the selected letter is “v”, what is the conditional
probability that the word “Dave” was selected in the first
place?
𝑃(Dave ∩ v)
𝑃 𝐷𝑎𝑣𝑒 𝑃(𝑣|𝐷𝑎𝑣𝑒)
𝑃 𝐷𝑎𝑣𝑒 𝑣 =
=
𝑃(𝑣)
0.15
1 ×1
4 = 0.556.
= 3
0.15
55