Download Probability and discrete Probability distributions

Chapter 6
Introduction to
Probability
1
Introduction
• In this chapter we discuss the likelihood of
occurrence for events with uncertain outcomes.
• We define a scale to measure and describe the
chance that different outcomes of an uncertain
event will take place.
• This ‘measure of uncertainty’ called “Probability”
serves as the basis for the analysis and results
discussed at the rest of this course.
2
6.1 Assigning probabilities to Events
• Random experiment
– a random experiment is a process or course of action,
whose outcome is uncertain.
• Examples
Experiment
Outcomes
• Flip a coin
• The marks of a statistics test
• The time to assemble
a computer
Heads and Tails
Numbers between 0 and 100
Non-negative numbers
3
6.1 Assigning probabilities to Events
• Because of the random nature of the experiment, repeated experiments
may result in different outcomes. Consequently, one cannot refer to the
experiment outcome in certain terms, only in terms of the probability
of each outcome to occur.
• To determine the probabilities we need to define and list the possible
outcomes first. Such a list should be
– exhaustive (list of all the possible outcomes).
– the outcomes are mutually exclusive (outcome do not overlap).
• A list of outcomes that meet the two conditions above, is called a
sample space.
4
Sample Space: S = {O1, O2,…,Ok}
O1
O2
Sample Space
a sample space of a
random experiment
is a list of all possible
outcomes of the
experiment. The
outcomes must be
mutually exclusive and
exhaustive.
Simple events
Event
An event is any collection
of one or more simple events
The individual outcomes
are called simple events.
Simple events cannot be
further decomposed
into constituent
outcomes.
5
Sample Space: S = {O1, O2,…,Ok}
Our objective is to
determine P(A), the
probability that event A will
occur.
6
Sample Space: S = {O1, O2,…,Ok}
Example 1: Build the sample space for the two random
experiments described below
– Case 1: A ball is randomly selected from an urn containing
blue and red balls. An outcome is considered the ball color.
The sample space: {B, R}
– Case 2: Two balls are randomly selected from an urn that
contains blue and red balls. An outcome is considered the
colors of the two balls drawn.
The sample space: {BB, BR, RB, RR}
7
Sample Space: S = {O1, O2,…,Ok}
• Example 2: Build the sample space for the random
experiment defined by the random selection of two
numbers from the set 1, 2, 3, 4, 5 (a number cannot be
selected twice). An outcome is defined by the sum of
the two numbers.
1+3
1+5, 2+4
3+5
The sample space: {3, 4, 5, 6, 7, 8, 9}
1+2 1+4, 2+3;
2+5, 3+4
4+5
8
6.1 Assigning probabilities to Events
• Given a sample space S={O1,O2,…,Ok}, the following
characteristics for the probability P(Oi) of the
experimental outcome Oi must hold:
1.
0  P(Oi )  1 for each i
2.
n
 P(Oi )  1
i 1
• The probability of an event: The probability P(A) of
event A is the sum of the probabilities assigned to
the simple events contained in A.
9
Approaches to Assigning Probabilities
and interpretation of Probability
• Approaches
– The classical approach
– The relative frequency approach
– The subjective approach
10
Events and their Probabilities
• An event is a collection of sample points.
– Example 3: A dice is rolled once. The following events are
defined. Specify the sample points that belong to each event.
Event A = the number facing up is 6
Event B = The number facing up is odd
Event C = The number facing up is not greater than 4
Solution:
A = {6};
B = {1, 3, 5};
C = {1, 2, 3)
– Example 4: A dice is rolled once and the outcome is 3.
Which event takes place?
Solution:
Event B and event C take place because the
outcome ‘3’ belongs to both events.
11
Probability of an Event
Example 5:
A dice is rolled once. Each number is equally likely to
face up. Find the probabilities of the following events:
Event A = the number facing up is 6
Event B = The number facing up is odd
Event C = The number facing up is
not greater than 4
Solution:
P(A) = P{6} = 1/6;
P(B) = P(1 or 3 or 5) = 3/6
P(C) = P(1 or 2 or 3) = 3/6
12
Probability of an Event
Example 6:
A dice is rolled twice: Define the following events:
D = The numbers facing up are the same and even
E = The sum of the two numbers facing up is 5.
Define the sample space and calculate the probabilities of events
D and E.
Solution
The sample space: S = {(1,1); (1,2);….(6,5); (6,6)}
P(D) = P{(2,2) or (4,4) or (6,6)} = 3/36
P(E) = P{(1,4) or (2,3) or (3,2) or (4,1)} = 4/36.
Note: there are 36 points in the sample space.
13
6.2 Joint, Marginal, and Conditional
Probability
• Determining the probability of an event involves
the counting process of its simple events. Often
this may become quite cumbersome.
• Understanding relationships among events may
reduce the computational effort involved in
determining the probability of combined events.
• Combined events and there probabilities are
discussed next
14
Intersection and Joint Probability
• The intersection of event A and B is the event that
occurs when both A and B occur.
A
C
B
15
Intersection and Joint Probability
• The intersection of events A and B is denoted
by “A and B” (A  B ).
• The probability of the intersection of A and B is
called also the joint probability of A and B =
P(A and B).
16
Intersection and Joint Probability
• Example 9
– A potential investor examined the relationship
between the performance of mutual funds and
the school the fund manager earned his/her
MBA.
– The following table describes the joint
probabilities.
17
Probabilities of Joint Events
• Example 9 – continued
– The joint probability of
P(A1 and B1)
[Mutual fund outperforms…] and [Top 20 MBA …] = .11
– The joint probability of
[Mutual fund outperform…] and […not top 20 MBA …] = .06
Mutual fund outperforms Mutual fund doesn’t
the market
outperform the market
(B2)
(B1)
Top 20 MBA program
(A1)
.11
.29
Not top 20 MBA program (A2)
.06
.54
18
Probabilities of Joint Events
• Example 9 – continued
– The joint probability of
P(A1 and B1)
[Mutual fund outperform…] and […Top 20 MBA …] = .11
– The joint probability of
P(A2 and B1)
[Mutual fund outperform…] and […not from a top 20 …] = .06
Mutual fund outperforms Mutual fund doesn’t
the market
outperform the market
(B2)
(B1)
Top 20 MBA program
(A1)
.11
.29
Not top 20 MBA program (A2)
.06
.54
19
Marginal Probabilities
To better understand the concept of marginal
probabilities, observe first the following demonstration
Let us separate event C into
Event
intersect
two sub event:
“ACand
C”, with
and “Bboth
and event
C” A and B
A
C
B
20
Marginal Probabilities
The intersection events “A and C”, and “B and C” are
indicated as two triangles, and for clarity are separated
for a moment (click).
A
A and C
B and C
B
21
Marginal Probabilities
As the two intersection events are brought back to there
original location, it becomes clear (we hope) that the
probability of event C can be calculated as the sum of
the two joint probabilities.
A
C
B
P(C) = P(A and C) + P(B and C))
22
Marginal Probabilities
• Applying this concept to the table of joint probabilities,
we notice that marginal probabilities are determining by
adding joint probabilities across rows and columns.
Click to continue.
• These probabilities are computed by adding across the
rows and down the columns and appear in the margins
of the table. Watch.
23
Marginal Probabilities
Mutual fund
Mutual fund
Marginal
outperforms the doesn’t outperform Prob.
market (B1)
the market (B2)
P(Ai)
Top 20 MBA program (A1)
P(A1 and B1) +
P(A1 and B2)
= P(A1)
Not top 20 MBA program (A2)
P(A2 and B1) +
P(A2 and B2)
= P(A2)
Marginal Probability P(Bj)
24
Marginal Probabilities
Mutual fund
Mutual fund
Marginal
outperforms the doesn’t outperform Prob.
market (B1)
the market (B2)
P(Ai)
Top 20 MBA program (A1)
.11
+
.29
=
.40
Not top 20 MBA program (A2)
.06
+
.54
=
.60
Marginal Probability P(Bj)
25
Marginal Probabilities
Mutual fund
Mutual fund
Marginal
outperforms the doesn’t outperform Prob.
market (B1)
the market (B2)
P(Ai)
Top 20 MBA program (A1)
P(A1 and B1)
+
Not top 20 MBA program (A2) P(A2 and B1
=
Marginal Probability P(Bj)
P(B1)
P(A1 and B2)
+
P(A2 and B2
=
P(B2)
.40
.60
26
Marginal Probabilities
Mutual fund
Mutual fund
Marginal
outperforms the doesn’t outperform Prob.
market (B1)
the market (B2)
P(Ai)
Top 20 MBA program (A1)
.11
.29
.40
Not top 20 MBA program (A2)
.06
.17
.54
.83
.60
Marginal Probability P(Bj)
+
+
27
Conditional Probability
• Frequently, information about the occurrence of
event A changes the probability of event B.
Here is a short demonstration of
one possible such situation
28
Conditional Probability
The sample space is S.
S
The probability event B
takes place is roughly:
This is event B
A
B
{The area of B}
{The area of S}.
The probability event B takes place
given ‘A’ took place is roughly:
{The area of B}
{The area of A}.
Now, assume event ‘A’ takes
place first followed by event B.
Note that event B is contained
in event A.
Explanation:
Since it is known ‘A’ took place, we
can reduce the sample space from
‘S’ to ‘A’.
Obviously P(B) is not equal to P(B given A)
29
Conditional Probability
• The probability of an event given the information
about the occurrence of another event is called
conditional probability.
• Specifically, the conditional probability of event B
given that event A has occurred is calculated as
follows:
P(A and B)
P(B|A) =
P(A)
30
Conditional Probability
• Example 13
– Find the conditional probability that a randomly selected fund
is managed by a “Top 20 MBA Program graduate”, given that
it did not outperform the market.
Mutual fund
outperforms
the market
(B1)
• Solution
Mutual fund
Marginal
doesn’t
Prob.
outperform the
P(Ai)
market (B2)
Top 20 MBA program
(A1)
.11
.29
.40
Not top 20 MBA
program (A2)
.06
.54
.60
Marginal Probability
P(Bj)
.17
.83
31
Conditional Probability
• Example 13 - continued
– Find the conditional probability that a randomly selected fund
is managed by a “Top 20 MBA Program graduate”, given that
it did not outperform the market.
• Solution
P(A1 and B2) = .29
P(B2) = .83
This information
reduces the relevant
sample space to the
83% of event B2.
Mutual fund
outperforms
the market
(B1)
Mutual fund
Marginal
doesn’t
Prob.
outperform the
P(Ai)
market (B2)
Top 20 MBA program
(A1)
.11
.29
.29
.40
Not top 20 MBA
program (A2)
.06
.54
.60
Marginal Probability
P(Bj)
.17
.83
.83
32
Conditional Probability
• Example 13 (Solution – continued)
P(A1|B2) = P(A1 and B2) = .29 = .3949
P(B2)
.83
33
Dependent Events
• Before the new information becomes available the
probability for the occurrence of A1 is
P(A1) = 0.40
• After the new information becomes available P(A1)
changes to
P(A1 given B2) = .3949
• Since the occurrence of B2 has changed the probability
of A1, the two events are related and are called
“dependent events”.
34
Independent Events
• Two events A and B are said to be independent
if
or
P(A|B) = P(A)
P(B|A) = P(B)
• That is: The probability of one event is not
affected by the occurrence of the other event.
35
Dependent and independent events
• Example 13 – continued
– We have already seen the dependency between A1
and B2.
– Let us check A2 and B2.
• P(B2) = .83
• P(B2|A2)=P(B2 and A2)/P(A2) = .54/.60 = .90
– Conclusion: A2 and B2 are dependent.
36
Mutually Exclusive Events
• Two events are said to be mutually exclusive if the
occurrence of one precludes the occurrence of the other
one.
• If A and B are mutually exclusive, by definition, the
probability of their intersection is equal to zero.
• Example: When rolling a dice once the event “The
number facing up is 6” and the event “The number
facing up is odd” are mutually exclusive.
37
Union
• The union event of A and B is the event that
occurs when either A or B or both occur.
• It is denoted “A or B” (A U B).
A
C
B
38
Union
• Example 9 – continued
Calculating P(A or B))
– Determine the probability that a randomly selected
fund outperforms the market or the manager
graduated from a top 20 MBA Program.
39
Union
• Solution
Mutual fund
outperforms the
market (B1)
Mutual fund
A1 or B1 occurs
doesn’t
outperform the whenever either:
A1 and B1 occurs,
market (B2)
Top 20 MBA program (A1)
.11
.29
A1 and B2 occurs,
Not top 20 MBA program (A2)
.06
.54
A2 and B1 occurs.
P(A1 or B1) = P(A1 and B1) + P(A1 and B2) + P(A2 and B1) =
.11 +.29 + .06 = .46
40
6.3 Probability Rules and Trees
• We present more methods to determine the
probability of the intersection and the union of
two events.
• Three rules assist us in determining the
probability of complex events.
– The complement rule
– The multiplication rule
– The addition rule
41
Complement of an Event
The complement of event A (denoted by AC) is
the event that occurs when event A does not
occur.
A
AC
42
Computing Probability using the
Complement
• The probability of event A can be calculated
using the probability of its complement event by
the complement rule
P(A) = 1 - P(AC)
Why is this true? Because A and AC consist of
all the simple events in the sample space.
Therefore, P(A) + P(AC) = 1
43
Multiplication Rule
• For any two events A and B
P(A and B) = P(A)P(B|A)
= P(B)P(A|B)
• When A and B are independent P(B|A) = P(B), so
P(A and B) = P(A)P(B)
44
Multiplication Rule
• Example 14
What is the probability that two female students will be
selected at random to participate in a certain research
project, from a class of seven males and three female
students?
• Solution
– Define the events:
A – the first student selected is a female
B – the second student selected is a female
– P(A and B) = P(A)P(B|A) = (3/10)(2/9) = 6/90 = .067
45
Multiplication Rule
• Example 15
What is the probability that a female student will be selected
at random in each of two classes of seven males and three
female students?
• Solution
– Define the events:
A – the student selected in one class is a female
B – the student selected in the other class is a female
– P(A and B) = P(A)P(B) = (3/10)(3/10) = 9/100 = .09
46
Addition Rule
For any two events A and B
P(A or B) = P(A) + P(B) - P(A and B)
Skip
P(A) =6/13
+
A
P(B) =5/13
_
P(A and B) =3/13
P(A or B) = 8/13
B
47
Addition Rule for Mutually Exclusive
Events
When A and B are mutually exclusive,
P(A or B) = P(A) + P(B) – P(A and B)
A
B
P(A and B) = 0
48
Addition Rule
• Example 11
– The circulation departments of two newspapers in a
large city report that 22% of the city’s households
subscribe to the Sun, 35% subscribe to the Post,
and 6% subscribe to both.
– What proportion of the city’s household subscribe to
either newspaper?
49
Addition Rule
• Solution
– Define the following events:
• A = the household subscribes to the Sun
• B = the household subscribes to the Post
– Calculate the probability
P(A or B) = P(A) + P(B) – P(A and B) = .22+.35 -.06 = .51
50
Addition Rule
• Example 12 (repeat of example 9)
Calculating P(A or B) using the addition rule
Determine the probability
that a randomly selected fund
outperforms the market or the
manager graduated from a
top 20 MBA Program.
• Solution
P(A1 or B1) =
P(A1)+P(B1)-P(A1 and B1)=
.4 + .17 - .11 = .46
Mutual
fund
outperfor
ms the
market
(B1)
Mutual fund Margi
doesn’t
nal
outperform Prob.
the market
P(Ai)
(B2)
Top 20 MBA
program (A1)
.11
.29
.40
Not top 20 MBA
program (A2)
.06
.54
.60
Marginal
Probability P(Bj)
.17
.83
51
Addition, Multiplication, and the
complementary rules
•
•
Example
A local photocopy shop has three black-and white
(BW) copy machines and two color copiers (CC). It is
known that a BW is down 10% of the time for repairs
and a CC is down 20%. Assume machines break
down independently.
–
What proportion of the time a customer cannot run a color
photocopy job?
Answer: The probability both CCs are down is (.2)(.2) = .04
52
Addition, Multiplication, and the
complementary rules
•
Example - Continued
–
If a customer wants both a color copy and a black and
white copy now, what is the probability he can complete
the two jobs?
Answer:
Each job can be completed if at least one machine of each
type is operational.
•
•
•
P(At least one CC is working) = 1 – P(Both CCs are down) =
1 – (.2)2 = .96.
P(At least one BW is working) = 1 – (.1)3 = .999
P(At least one CC and at least one BW is working) =
P(At least one CC is working)*P(At least one BW is working) =
(.96)(.999).
53
Addition, Multiplication, and the
complementary rules
•
Example - Continued
–
If a customer wants both a color copy and a black and
white copy now, but this time a CC can perform also a BW
job, what is the probability he can complete the two jobs?
Answer:
P(The two jobs can be completed) =
P(The two CCs are working or one CC and at least one
BW are working) = P(The two CCs are working)+
P(One CC is working)*P(At least one BW is working) =
2(.8)(.2) + [1 – (.1)3]
54
Probability Trees
• This is an effective graphical tool that help apply
probability rules.
• The events are represented by the tree
branches.
• The given probabilities are indicated along these
branches too.
55
Probability Trees
• Example 14 revisited (dependent events).
– Find the probability of selecting two female students (without
replacement), if there are 3 female students in a class of 10.
Joint probabilities
FF P(FF)=(3/10)(2/9)
Second selection
First selection
FM P(FM)=(3/10)(7/9)
MF P(MF)=(7/10)(3/9)
Second selection
MM P(MM)=(7/10)(6/9)
56
Probability Trees
• Example 15 – revisited (independent events)
– Find the probability of selecting one female student in each of
two classes if there are 3 female students in a class of 10.
FF P(FF)=(3/10)(3/10)
Second
selection
FM P(FM)=(3/10)(7/10)
MF P(MF)=(7/10)(3/10)
First
selection
Second
selection
MM P(MM)=(7/10)(7/10)
57
Probability Trees
• Example 16 (conditional probabilities)
– The pass rate of first-time takers for the bar exam at
a certain jurisdiction is 72%.
– Of those who fail, 88% pass their second attempt.
– Find the probability that a randomly selected law
school graduate becomes a lawyer (candidates
cannot take the exam more than twice).
58
Probability Trees
• Solution
P(Pass1) = .72
.72
+
.2464
First
exam
Second
exam
P(Fail1 and Pass2)=.
.28(.88)=.2464
P(Fail1 and Fail2) =
(.28)(.12) = .0336
P(Pass) = P(Pass on first exam) + P(Fail on first and Pass on second) = 59
.9664
Baye’s Law
• We use Baye’s law to find the conditional
probability of a possible cause for an event
that is known to have occurred.
60
Baye’s Law
If event B took place,
either event A or AC
could have occurred
before.
Baye’s law calculates
the conditional probability
P(A|B) and P(AC|B).
61
Baye’s Law
P( A and B)
P( A | B) 
P(B)
62
Baye’s Law
• Example 17
– Medical tests can produce false-positive or false-negative
results.
– A particular test is found to perform as follows:
• Correctly diagnose “Positive” 94% of the time.
• Correctly diagnose “Negative” 98% of the time.
– It is known that 4% of men in the general population suffer from
the illness.
– What is the probability that a men is suffering from the illness, if
the test result were positive?
63
Baye’s Law
• Solution
– Define the following events
•
•
•
•
D = Has the disease
DC = Does not have the disease
PT = Positive test results
NT = Negative test results
Called before: A
Called before: AC
Called before: B
– Build a probability tree
64
Baye’s Law
• Solution – Continued
– The probabilities provided are:
• P(D) = .04
•
P(NT|DC)
P(DC) = .96
= .98
• P(PT|D) = .94
P(PT|DC)
= .02
P(NT|D)= .06
4% of the population has the disease.
The test result is “Negative” 98% of
the time given that a patient is not ill.
The test result is “Positive” 94% of
the time given that a patient is ill.
• The probability to be determined is
P(D | PT )
The patient is ill given that the test
comes “Positive”.
65
Baye’s Law
P(D and PT)
=.0376
+
P(D | PT ) 
P(DC and PT)
=.0192
P(D and PT)
P(PT)
.0376

 .6620
.0568
P(PT) =.0568
P(PT) = P(D and PT) + P(DC and PT)
66
Baye’s Law
Prior
probabilities
Likelihood
probabilities
Posterior probabilities
P(D | PT )  .0376  .6620
.0568
The prior probability of event ‘D’ was 4%,
but because event ‘PT’ occurred the
probability the cause had been ‘D’
increased to 66.2%.
67