Download Conditional Probability

Conditional Probability
STA 281 Fall 2011
1 Definition
Often we are only interested in particular rows or columns of a probability table. Consider the
newspaper example, and the question “Of those that receive the morning paper, what proportion receive
the evening paper?” This question does not concern the entire population of households; it only
concerns those who receive a morning paper. Probabilities that refer only to subsets of the population
are called conditional probabilities. Recall the probability table we constructed
0.10 0.20 0.30
0.50 0.20 0.70
0.60 0.40 1.00
The question asked concerns only those who receive a morning paper, which is 60% of the entire
population. We want to know, out of that 60%, what proportion receive the evening paper. To provide
an intuitive fell for how this question is answered, suppose we sampled 100 people from the population.
On average 60 of those people would receive the morning paper. Looking at the M column of the table,
we see that of those 60, on average 50 receive only the morning paper while 10 receive both. So 10 of
the 60 who receive the morning paper also receive the evening paper, so the conditional probability is
10/60=1/6.
Usually we don’t go through the argument concerning sampling a set of people and just divide the
probabilities directly. There are 60% of the people who receive a morning paper, with 50% receiving
only the morning paper and 10% receiving the evening paper. So 0.10/0.60=1/6 is the conditional
probability of receiving an evening paper given one receives a morning paper.
Mathematically, a conditional probability has two parts: First, a conditional probability only asks
about a subset of the population, not the entire population. Second, a conditional probability asks some
property of that subset. In our example question, the subset of interest was those who receive the
morning paper, while the property we are interested in was receiving an evening paper. In general, we
have a question: “of those who are in subset A, what is the probability they are in B.” This question is
translated into mathematical symbols
, which is read “the probability of B given A.”
Notice how we solved the problem. First, we found which of the individuals were in the subset of
interest. This involved finding
, the unconditional probability of the subset. Then, within that
subset, we found how many individuals had the property we were interested in. The result was
Instead of writing this fraction in words, we can use symbols. For the denominator, the “people in
subset A” refers to
. For the numerator, the people must be in subset A, but they must also have
property B. Since both criteria must be satisfied, the numerator is
, resulting in
Mathematically, this formula is the definition of conditional probability.
immediately implies what is called the intersection rule
1
Rearranging the terms
Similarly, since just switching the roles of A and B in the definition of conditional probability yields
, we find
Since
is the same as
, the two previous equations provide two ways of finding the
probability of an intersection. Simply use whichever conditional probability is more convenient.
2 Recognizing Conditional Probabilities
Remember the key point about conditional probability is that we are only interested in a subset of the
population. The first step in evaluating a conditional probability is determining which outcomes we are
interested in. In our example we were interested in people who received a morning paper. The second
step is to identify the property of interest (in our example it was receiving an evening paper). After that
you use the definition of conditional probability
.
Whenever you see a probability stated or are asked for a probability, ask yourself two questions.
First, “Who is this statement about?” In a conditional probability we are only interested in a subset of
the population. It is important to determine which subset as soon as possible so we can proceed.
Second, “What are they asking about?” That is, what property of the subset is the question or statement
about.
For example, suppose our population is all registered voters. Contrast the two questions: “What
proportion of women are Democrats?” and “What proportion of voters are women Democrats?” The
first statement does not ask about all registered voters, it only asks about women. Therefore it is a
conditional probability. After deciding it only asks about women, we must then determine what exactly
it wants to know about women. In this example, the property of interest is being Democratic. If W is
the event “voter is female” and D is the event “voter is a Democrat”, the first question asks for
.
The second question does not place any restriction on the population since it asks for the proportion of
voters. The property it is interested in is whether a voter is a Democratic woman. Thus the second
question is asking for
. These are separate questions, so you must recognize which one you
are being asked.
Of course, in any language there are multiple ways to ask the same question. The following
questions are equivalent, all ask for the probability of receiving an evening paper in the set of outcomes
where a morning paper was received.

What proportion of those that receive a morning paper receive an evening paper?

Given a person receives a morning paper, what is the probability they receive an evening paper?

If someone receives a morning paper, what is the probability they receive an evening paper?
There are a number of ways to report the result P(A|M)=1/6 as well.

1/6 of those that receive a morning paper receive an evening paper.

Given someone receives a morning paper, there is a 1/6 probability of receiving an evening
paper.

If someone receives a morning paper there is a 1/6 probability of receiving an evening paper.
2
The probability table allows us to compute conditional probabilities fairly easily, since we are just
involved with one row or column of the table. More complicated conditional probabilities may be
computed as well. For example, given that someone receives at least one of the papers, what is the
probability they receive at most one of the papers? This is a conditional probability since we are
interested only in those who receive at least one paper, not everyone. This subset of individuals
includes 3 cells of our table, for a total of 80% of the population.
Within that subset, we are interested in the property “receive at most one of the papers”. As with
all conditional probabilities, we look at the individuals within the subset and try to determine which
obey the property. Just looking at the cells which compose the subset, we find only the cells
and
satisfy the property. Those two boxes total 70% of the overall probability. The
conditional probability is thus 0.70/0.80=7/8.
3 Using Conditional Probabilities
The definition of conditional probability is that
. We derive previously the
intersection rule
. This formula allows us to take conditional probabilities as
given information and complete a probability table.
Suppose in a small university, students work in either the dorms or library. 70% of the students do
some work in the library while 10% work only in the dorms. Of those who work in the library, 30% also
work in the dorms. Use this information to construct a probability table.
Students may either work in the dorms or the library. Since they may work or not work in the
library and work or not work in dorms, we may construct a probability table.
1.00
We are given the information that 70% of the students work in the library. Since this information was
given without any reference to whether or not those students work in the dorms, it is therefore placed in
the margins of the table. We are also given that 10% work only in the dorms. This is one of the 4 core
cells of the table. Before working with the conditional probability, we may use some arithmetic to fill in
some of the table.
0.10
0.20
0.30
0.70
1.00
To complete the table, we must use the conditional probability given in the problem. We are
given the information that of those who work in the library, 30% work in the dorms. This is a
conditional probability, not the proportion that work in both. We are conditioning on people who work
in the library (70% of the students). We are given that 30% of those 70% also work in the dorm. 30% of
70% may be found by multiplying the probabilities, so (0.30)(0.70)=0.21 work in the dorm and the
library. Mathematically, we have been given
=0.30, and we have found
We may complete the table.
Since we have computed the probabilities of all outcomes, we may compute any probabilities from
the table.
3

What is the probability a student works only in the library? (0.49)

What is the probability a student works in the dorms? (0.31)

What is the probability a student works in either the dorms or in the library?
(0.21+0.49+0.10=0.80)

What is the probability a student works in neither the library nor the dorms? (0.20)

What is the probability a student that does not work in the library works in the dorms?
(0.10/0.30=1/3)
4 More difficult problems
In the dorm and library example, the conditional probability provided allowed the direct computation of
a cell probability. Problems involving conditional probabilities may be more difficult.
4.1
Applicants Example
Suppose a company is looking at applicants for a position. The position requires (A) experience and (B)
a master’s degree. Suppose that 90% of the applicants have at least one of (A) or (B). Suppose further
that, of those applicants with at least one of (A) or (B), 50% have both. Of those applicants with exactly
one of (A) or (B), 2/3 have (A).
We can construct a probability table and fill in one of the cells directly. Since 90% of the applicants
have at least one of (A) or (B), we may determine by the complement rule that 10% have neither. This
results in the table
0.10
1.00
To fill in the remainder of the table, we have to use conditional probabilities. We are given that 50% of
the people with at least one have both. This probability involves the subset “with at least one”. This
subset contains 90% of the applicants. Of this 90%, we are given 50% have both. Thus 50% of the 90%,
or 45%, have both.
The remaining cells must be found by utilizing that 2/3 of the applicants with exactly one of (A) or
(B) have (A). Although we are not given the proportion of individuals with exactly one directly, we can
derive it to be 1-0.45-0.10=0.45. Since 2/3 of those 45% have (A), we conclude
and
complete the table through arithmetic.
4.2
Professors and their Stories Example
Here is another example. Suppose that 10% of professors can think of decent stories to go with their
exam problems. Of those that can, 60% can think of funny stories. Suppose also that, of those
professors whose stories are at least one of funny/decent, 75% of the professors write funny stories.
The first two probabilities allow us to complete part of the table.
0.06
0.04
0.10
0.90
4
1.00
To fill in the remaining entries, we must use the information that “of those professors whose stories are
at least one of funny/decent, 75% of the professors write funny stories”. Unfortunately, with the table
so far we cannot compute the probability of being at least one of funny or decent, because we are
missing the cell
. Let’s just fill in this unknown quantity with x and see if we can make
progress in solving for x. In terms of the unknown x, the proportion of professors whose stories are at
least one of funny or decent is 0.06+0.04+x=0.10+x. We are given
Solving for x, we find x=0.06, so we may complete the table. As with all probability tables, once the
table is completed we may compute any probability.

What proportion of professors write both decent and funny stories? (0.06)

Of the professors with funny stories, what proportion write decent stories? (0.06/0.12=0.5)

Of those professors whose stories are exactly one of funny or decent, what proportion write
funny stories? (0.06/0.10=0.6)
5 Independence
If
is different than
, then A and B are related. Return to the newspaper example. We
find
=1/6 while
=1/2. People who receive the morning paper are less likely to receive
the evening paper than people who do not receive the morning paper. Since the probability the event E
occurs depends on whether or not M occurred, we call these events dependent. There are many events
of this form. Any events with a causal relationship, for example, will be of this form. Individuals with a
Ph.D. are much more likely to get a faculty position at a research university than individuals who do not
have a Ph.D., the simple reason being a Ph.D. is a requirement for such a faculty position. Indirect links
between variables also create dependence. An elementary school student with large feet is more likely
to read well. Foot size and reading ability have no causal link, but children with big feet tend to be
older, and older children tend to read better than younger children. All of these types of relationships
between variables result in dependent events. Much of science is concerned with dependence. In
medicine, for example, we would like to know whether administering a treatment to a patient increases
the patient’s chance of recovery.
Occasionally two variables are unrelated. A coin has no memory. If you flip a fair coin twice, the
result of the second flip has nothing to do with the result of the first flip. The probability of heads is
always 0.5. If
is the event “heads on the first flip” and H2 is the event “heads on the second flip”, then
=0.5 and
=0.5. They are the same probability since the result of the first flip does
not affect the probability of heads on the second flip.
Events A and B where
are called independent events. Essentially, whether or
not B occurred does not affect the probability A occurred. The purpose of this section is to derive some
special properties of independent events. With the information
, we can derive some
equivalent definitions of independence. Recall the Law of Total Probability
Using the rule for intersections
5
Now using the assumed information
[
]
This equivalent definition of independence states that the overall probability of A, P(A), is the same as
the conditional probability of A given B, P(A|B). If you know the overall probability of A, knowing
whether or not B occurred does not change that probability. Another equivalent definition may be
found by using the rule for intersections
This third definition is the one used for mathematical purposes. The reason is that the conditional
probability is a fraction, and thus there is the possibility of dividing by zero. The equation above
presents no such problem.
If P(A) and P(B) are both greater than 0, then all three definitions are equivalent. However, if
P(A)=0 or P(B)=0, then the first two definitions might result in division by zero problems. To verify
independence in this course, you must show the third definition, that
.
For example, reconsider the newspaper example.
independent?” you should check
If you were asked “Are events M and E
Since the equality required for independence does not hold, the events are dependent, not independent.
Alternatively, suppose a probability table has
0.08
0.32
0.40
0.12
0.48
0.60
0.20
0.80
1.00
In this example,
and thus the events are independent.
6 The Relationship between Disjoint and Independent
The short answer is that there is no relationship between two disjoint events and two independent
events. They are separate definitions, and they are useful in different scenarios.
Disjoint events are defined as events A and B such that
. Disjoint events are useful in
that they simplify the rule for unions. In general, the rule for unions states
If A and B are disjoint, then
and thus the union rule simplifies to the third axiom
6
Remember, you need the disjoint assumption to make the simplification (technically you only need
, which you can derive from the disjoint assumption).
Independence is defined as
intersections. In general,
, and thus results in a simplification of the rule for
If A and B are independent, this simplifies to
Remember you have to have the independence assumption to make this simplification.
Independent events are sometimes disjoint and sometimes not, while disjoint events are sometimes
independent and sometimes not. You have to know your purpose and check the appropriate definition.
7 Bayes Rule
A conditional probability concerns a subset of the population. Within that population, all the axioms of
probability apply. For example, all conditional probabilities have to be nonnegative, just like axiom 2.
Theorems such as the complement rule also still apply. Thus, we can prove theorems like
Suppose a university is interested in graduation rates between students who have off-campus jobs and
students who do not. Let G be the event “graduates in 6 years or less” and W be the event “works offcampus”. If you are given the information that 60% of the students with off-campus jobs graduate in 6
years, then P(G|W)=0.60. You can also conclude that 40% of the students with off-campus jobs do
NOT graduate. In symbols, you can conclude P(GC|W)=0.40. In both conditional probabilities, you are
conditioning on the same group of people, students with off-campus jobs. If 60% of those students
graduate, the other 40% of those students do not.
Unfortunately, many people make the mistake of doing calculations based on different populations,
such as concluding, incorrectly, that P(A|B)=1-P(A|BC). In our example, this would be the same as
concluding that since 60% of the students with off-campus jobs graduate, then only 40% of the students
without off-campus jobs graduate. But this conclusion is unwarranted. Just because 60% of the students
with off-campus jobs graduate doesn’t say anything about the students without off-campus jobs. They
are separate groups of people, and can have separate, unrelated probabilities. It’s possible all students
without off-campus jobs graduate. It is also possible none of them do, or anywhere in between.
Another common mistake is to assume P(A|B)=1-P(B|A). In our example, this would be the same
as using the information that 60% of students with off-campus jobs graduate to conclude that 40% of
students who graduate have off-campus jobs. This conclusion is unwarranted as well. To make this
more obvious, let F be the event a person is female and P be the event the person is pregant. Suppose at
any given time that 2% of women are pregnant. This says P(P|F)=0.02. You cannot then conclude
that 98% of pregnant people are female.
There actually is a relationship between P(A|B) and P(B|A), but it is more complicated. Note by
definition
7
By the law of total probability
, so
We can then use the rule for intersections repeatedly to show
This last equation is a simplified version of Bayes rule, which is vital in many scientific applications.
Typically the event A is that some hypothesis is true, and thus AC is the event the hypothesis is false.
The event B corresponds to some piece of data being observed. By comparing the relative likelihoods A
and AC assign to B (i.e., the relative probability of observing the data when the hypothesis is true P(B|A)
versus observing the data when the hypothesis is false P(B|AC) you can compute how observing the data
B affects your belief in the hypothesis A.
Note that Bayes rule includes the probabilities P(A) and P(AC). This is useful in situations such as
medical diagnostic testing, where it is worthwhile to incorporate relative rates of disease into the
calculations. If an exotic, rare disease produces a particular symptom but a common disease also
produces the symptom, when faced with the symptom the more likely outcome is that the common
disease is present.
In this course, Bayes rule is typically not computed symbolically, but arises naturally as you fill in a
probability table. Conditional probabilities allow you to compute cells of the table, and after the table is
completed you can compute whatever probability you wish.
8 Sample Questions
The following questions are divided into three levels of difficulty (all relative). The “easiest” problems
contain no conditional probabilities in the given information, but do ask you to compute conditional
probabilities from the table. The “medium” problems have conditional probabilities in the given
information as well, so you must use the conditional probabilities to construct the table. The “hardest”
problems often require some type of algebra to construct the probability table.
1) (Easiest) In a study of 3756 court cases, Kalven and Zeisell (1966) recorded the jury panel’s
decision. In addition, they separately asked the judge how he or she would have decided the same
case if there were no jury panel. They found that: the judge would not have convicted in 17% of the
cases; both the judge and the jury would have convicted in 64% of the cases; and the judge and jury
disagree on whether to convict in 22% of the cases
a) Construct a probability table.
b) What is the probability that neither the judge nor the jury would convict?
c) Given the panel convicts, what is the probability the judge would also convict?
d) Are the events “judge convicts” and “jury panel convicts” disjoint? Why or why not?
8
2) (Easiest) Suppose in a particular company, employees use workstations or PCs (could be both or
neither). The probability an employee uses a PC is 0.95 and the probability an employee uses a
workstation is 0.20. The probability an employee uses both is 0.19.
a) Construct a probability table.
b) What is the probability an employee who does not use a PC uses a workstation?
c) What is the probability an employee uses exactly one of the machines?
d) Given an employee uses at least one of the machines, what is the probability they use a
workstation?
3) (Easiest) A muscle cell has 2 sites where electricity can conduct into the cell. Every time the
body intends to stimulate the muscle cell, an attempt is made to channel electricity through each of
the 2 sites. The probability both sites conduct electricity is 0.4. The probability that exactly one
site conducts electricity is 0.4. Finally, the probability site 1 conducts electricity is 0.7.
a) Construct a probability table.
b) What is the probability site 2 conducts electricity?
c) Given at least one of the sites conducts electricity, what is the probability both sites conduct
electricity?
4) (Medium) Suppose a professor only writes two types of exams, “easy” or “hard”. Suppose that 90%
of the exams are hard. There is an 80% chance that the first question on a hard exam will be
difficult, and a 15% chance that the first question on an easy exam will be difficult.
a) Construct a probability table.
b) On a given exam, what is the probability it is a hard exam that contains a non-difficult first
question?
c) On a given exam, what is the probability that the first question will be non-difficult?
d) Suppose the first question on a given exam is non-difficult. Given this information, what is the
probability the exam is hard?
e) Are the events “easy exam” and “difficult first question” independent? Why?
5) (Medium) Suppose at a particular park visitors can hike or raft. The probability that someone will
raft is 0.40. The probability that someone won’t hike is 0.10. Given someone hikes, the probability
they raft is 0.40.
a) Construct a probability table.
b) What is the probability of rafting?
c) What is the probability of not hiking?
d) What is the probability of someone participating in exactly one?
e) What is the probability of someone participating in at least one?
f)
Of those who do not participate in both, what proportion participate in neither?
9
6) (Medium) Suppose that 10% of all cases go to trial. Of those that go to trial, the defendant is
found guilty in 95% of the cases. Of those that do not go to trial, the defendant is found guilty
(through a plea bargain) in 40% of the cases.
a) Construct a probability table.
b) What proportion of defendants are found guilty?
c) Given a defendant is found guilty, what is the probability their case went to trial?
d) What proportion of cases that go to trial do not result in a guilty plea?
7) (Hardest) Investment advisors might subscribe to the Wall Street Journal or Investor’s Business
Daily. Suppose 80% subscribe to the WSJ. Of those that subscribe to the WSJ, 75% also subscribe
to IBD. Also, 20% of those that subscribe to exactly one of the papers subscribe to IBD.
a) Construct a probability table.
b) What is the probability an investment advisor subscribe to neither paper?
c) What proportion of investment advisors subscribe to IBD?
d) Given an investment advisor receives IBD, what is the probability they also receive the WSJ?
8) (Hardest) Suppose there are two restaurants in a small town, Abelard’s Attic and Baltazar’s Buffet.
Suppose further that 20% of the people in the town dine at neither restaurant. Of those that go to at
least one of the restaurants, 75% dine at both. Of those that dine at exactly one of the restaurants,
75% dine at Abelard’s.
a) Construct a probability table.
b) What proportion dine at Baltazar’s?
c) Of those that dine at Baltazar’s, what proportion dine at Abelard’s?
9) (Hardest) Let A and B be events. Suppose that P(A)=0.3 and P(Bc|A)=0.2. Suppose further that
given exactly one of the two events occurs, 40% of the time it is B that occurred.
a) Construct a probability table.
b) Given that at least one of the events occurred, what is the probability B occurred?
c) What is P(B|A)?
10) (Hardest) Let A and B be events. Suppose
. Suppose also that, conditional on
exactly one of the events occurring, the probability A occurs is 0.8. Finally, suppose that the
probability neither event occurs given B did not occur is 1/9.
a) You should know what to do for part (a) by now.
b) What is the probability that at least one of the events occurs?
c) Given that at most one of the events occur, what is the probability A occurs?
10
9 Solutions for Sample Problems
1)
a)
P
Pc
J
0.64
0.19
0.83
Jc
0.03
0.14
0.17
0.67
0.33
1.00
b) 0.14
c) 0.64/0.67=0.9552
d) They are not disjoint, they occur together with probability 0.64.
2)
a)
WS
WSc
PC
0.19
0.76
0.95
PCc
0.01
0.04
0.05
0.20
0.80
1.00
b) 0.01/0.05=0.20
c) 0.76+0.01=0.77
d) 0.20/(0.19+0.01+0.76)=0.2083
3)
a)
Site 1 conducts
0.40
0.30
0.70
Site 2 conducts
Site 2 doesn’t conduct
Site 1 doesn’t conduct
0.10
0.20
0.30
0.50
0.50
1.00
b) 0.50
c) 0.40/0.80=0.5
4) Let E be the event an exam is easy. Let D be the event the first question is difficult.
a)
D
Dc
E
0.015
0.085
0.100
Ec
0.720 0.735
0.180 0.265
0.900 1.00
b) 0.180
c) 0.265
d) 0.180/0.265=0.6792
11
e) The events are dependent, since
5)
a)
R
Rc
H
0.36
0.54
0.90
Hc
0.04
0.06
0.10
0.40
0.60
1.00
b) 0.40
c) 0.10
d) 0.54+0.04=0.58
e) 0.36+0.54+0.04=0.94
f)
0.06/0.64=0.0938
6) Let T be the event “go to trial” and G be the event “found guilty”.
a)
G
Gc
T
0.095
0.005
0.100
Tc
0.360 0.455
0.540 0.545
0.900 1.00
b) 0.455
c) 0.095/0.455=0.2088
d) 0.005/0.100=0.05
7)
a)
IBD
IBDc
WSJ
0.60
0.20
0.80
WSJc
0.05 0.65
0.15 0.35
0.20 1.00
b) 0.15
c) 0.65
d) 0.60/0.65=0.9231
12
8)
a)
B
Bc
A
0.60
0.15
0.75
Ac
0.05
0.20
0.25
0.65
0.35
1.00
B
Bc
A
0.24
0.06
0.30
Ac
0.04
0.66
0.70
0.28
0.72
1.00
B
Bc
A
0.45
0.40
0.85
Ac
0.10
0.05
0.15
0.55
0.45
1.00
b) 0.65
c) 0.60/0.65=0.9231
9)
a)
b) 0.28/0.34=0.8235
c) 0.24/0.30=0.8
10)
a)
b) 0.95
c) 0.40/0.55=0.7273
13