Download Lecture 7: Bayes` Formula 1. Conditional Probabilities as a

Lecture 7: Bayes’ Formula
1. Conditional Probabilities as a Computational Tool:
Let E and F be two events and notice that we can write E as the following disjoint union:
E = (E ∩ F ) ∪ (E ∩ F c ).
Then, using the additivity of probabilities and the definition of conditional probabilities, we
obtain:
P(E) = P(E ∩ F ) + P(E ∩ F c )
= P(E|F )P(F ) + P(E|F c )P(F c )
= P(E|F )P(F ) + P(E|F c )(1 − P(F )).
This formula can often be used to calculate the probability of a complicated event by exploiting
the additional information provided by knowing whether or not F has also occurred. The trick
is to know how to choose F so that P(E|F ) and P(E|F c ) are both easy to calculate.
Example (Ross, 3c): Suppose that there are two classes of people, those that are accident
prone and those that are not, and that the risk of having an accident within a one year period
is either 0.4 or 0.2 depending on which class a person belongs to. If the proportion of accidentprone people in the population is 0.3, calculate the probability that a randomly selected person
has an accident in a given year (0.26). What is the conditional probability that an individual is
accident-prone given that they have had an accident in the last year? (6/13).
Example (Ross, 3d): Suppose that a blood test has a 95% chance of detecting a disease when
it is present, and that it has a 1% rate of false positives. If 0.5% of the population has the
disease, then the probability that a person has the disease given that their test result is positive
is only 0.323.
2. The previous result can be generalized in the following way. Suppose that F1 , · · · , Fn are
mutually exclusive events such that E ⊂ F1 ∪ · · · ∪ Fn . Then, because the sets E ∩ Fi and E ∩ Fj
are disjoint whenever i 6= j and because
E=
n [
E ∩ Fi ,
i=1
we know that
P(E) =
n
X
P(E ∩ Fi )
i=1
=
n
X
P(E|Fi )P(Fi ).
i=1
1
Remark: The first identity is sometimes known as the Law of Total Probability.
3. Bayes’ formula tells us how to calculate the conditional probability of one of the events Fj
given that the event E is known to have occurred:
P(Fj |E) =
P(E ∩ Fj )
P(E|Fj )P(Fj )
.
= Pn
P(E)
i=1 P(E|Fi )P(Fi )
Example(Ross, 3l): Suppose that we have three cards, and that the first card has two red
sides, the second card has two black sides, and the third card has one red side and one black
side. One of these cards is chosen at random and placed on the ground. If the upper side of this
card is red, show that the probability that the other side is black is 1/3.
Example (Ross, 3n): A bin contains three types of flashlights (types 1, 2 or 3) in proportions
0.2, 0.3 and 0.5, respectively. Suppose that the probability that a flashlight will give over 100
hours of light is 0.7, 0.4, and 0.3 for type 1, type 2 or type 3.
a) Show that the probability that a randomly chosen flashlight gives over 100 hours of light is
0.41.
b) Given that a flashlight gives over 100 hours of light, show that the probability that it is a
type j flashlight is 14/41 for type 1, 12/41 for type 2 and 15/41 for type 3.
Example (problem 3.26): Suppose that 5% of men and 0.25% of women are color blind. A
color-blind person is chosen at random. Assuming that there are an equal number of males and
females in the population, what is the probability that the chosen individual is male?
Answer: Let M and F be the event that the chosen individual is a male or female, respectively,
and let C denote the event that a randomly chosen individual is color blind. Then the probability
of interest is
P(C|M )P(M )
P(C|M )P(M ) + P(C|F )P(F )
0.05 ∗ 0.5
=
0.05 ∗ 0.5 + 0.0025 ∗ 0.5
= 0.9524.
P(M |C) =
2
4. Bayesian statistics:
Bayes’ formula is often applied to problems of inference in the following way. Suppose that
our goal is to decide between a set of competing hypotheses, H1 , · · · , Hn . Let p1 , · · · , pn be
a probability distribution on the hypotheses, i.e., p1 + · · · + pn = 1, where pi is a measure of
the strength of our belief in hypothesis Hi prior to performing the experiment. In the jargon of
Bayesian statistics, (p1 , · · · , pn ) is said to be the prior distribution on the space of hypotheses.
Now suppose that we perform an experiment to help decide between the hypotheses, and let the
outcome of this experiment be the event E. Bayes’ formula can be used to update our belief
in the relative probabilities of the different hypotheses in light of the new data. The posterior
probability, p∗i , of hypothesis Hi is defined to be the conditional probability of Hi given E:
P(E|Hi )P(Hi )
pi P(E|Hi )
p∗i = P(Hi |E) = Pn
= Pn
.
j=1 P(E|Hj )P(Hj )
j=1 pj P(E|Hj )
Notice that (p∗1 , · · · , p∗n ) is a probability distribution on the space of hypotheses that is called
the posterior distribution. It depends both on the data collected in the experiment as well
as on our prior belief about the likelihood of the different hypotheses.
Example: Suppose that we have a population containing n individuals and our goal is to
estimate the number of female members of this population. Let Hk be the hypothesis that
the population contains k females and n − k males, and suppose that our prior distribution on
(H0 , · · · , Hn ) is given by
n 1 n
pk =
, k = 0, 1, · · · , n.
k 2
(This prior distribution would be justified, for example, if we believed that each member of the
population was equally likely to be a female or a male.)
Now, suppose that the experiment consists of sampling one member of the population and
determining their gender. Let E be the event that that individual is female and observe that
P(E|Hk ) = k/n and that
P(E) =
n
X
pk P(E|k) =
k=0
n X
n 1 nk
k=0
k
2
n
=
1
.
2
Using Bayes’ formula, the posterior probability of hypothesis Hi is
pi P(E|Hi )
n 1 ni
n − 1 1 n−1
∗
pi =
= 2
=
1/2
i
2 n
i−1
2
for i = 1, · · · , n, while p∗0 = 0. Notice that p∗i ≥ pi for i ≥ n/2, i.e., observing a female in our
sample increases the posterior probability that the population sex ratio is female-skewed.
3
5. Mendelian genetics:
Key facts:
• Humans have two copies of most genes (except on the sex chromosomes).
• A newborn child independently inherits one gene from each parent, i.e., one from the
mother and one from the father.
• Each of the two gene copies in a parent is equally likely to be inherited by a newborn child.
• Different types of a gene are called alleles. These are often denoted by capital and
lowercase letters, e.g., B and b.
• A genotype is a list of the types of the two genes possessed by an individual.
• If there are two alleles, then there are three possible genotypes: BB, bb, and Bb. Notice
that we usually ignore the order of alleles when writing the genotype, i.e., Bb and bB are
treated as the same genotype.
• The types and frequencies of the genotypes inherited by offspring can be calculated using
a Punnett square. With two alleles, B and b, the possible crosses are:
– BB × BB produces only BB
– bb × bb produces only bb
– BB × Bb produces 21 BB + 21 Bb
– bb × Bb produces 12 bb + 12 Bb
– Bb × Bb produces 14 BB + 12 Bb + 14 bb
– BB × bb produces only Bb
Example: Eye color is determined by a single pair of genes, which can be either B or b. If a
person has one or more copies of B, then they will have brown eyes. If a person has two copies
of b, then they will have blue eyes. Suppose that Smith and his parents have brown eyes, and
that Smith’s sister has blue eyes.
a) What is the probability that Smith caries a blue-eyed gene?
b) Suppose that Smith’s partner has blue eyes. What is the probability that their first child will
have blue eyes?
c) Suppose that their first child has brown eyes. What is the probability that their second child
will also have brown eyes?
Answer:
a) Because Smith’s sister has blue eyes, her genotype is bb. Consequently, both of Smith’s
parents (who are brown-eyed) must have genotypes Bb, since the sister must inherit a b gene
from each parent. However, Smith himself could have genotype BB or Bb. Let E1 and E2 be
4
the events that Smith has genotype BB or Bb, respectively, and let F be the event that Smith
has brown eyes. Then the probability that Smith has genotype Bb is
P(F |E2 )P(E2 )
P(F )
1 · 1/2
=
3/4
= 2/3,
P(E2 |F ) =
while the probability that Smith has genotype BB is P(E1 |F ) = 1 − P(E2 |F ) = 1/3.
b) If Smith’s partner has blue eyes, then her genotype is bb. Now if Smith has genotype BB,
then his children can only have genotype Bb and therefore have zero probability of having blue
eyes. On the other hand, if Smith has genotype Bb, then his first child will have blue eyes with
probability 1/2 (i.e., with the probability that the child inherits a b gene from Smith). Thus,
the total probability that the first child has blue eyes is:
1/3 · 0 + 2/3 · 1/2 = 1/3.
c) Knowing that their first child has brown eyes provides us with additional information about
the genotypes of the parents. Let F be the event that the first child has brown eyes and let E1
and E2 be the events that Smith has genotype BB or Bb, respectively. From (a) we know that
P(E1 ) = 1/3 and P(E2 ) = 2/3, while (b) tells us that P(F ) = 1 − 1/3 = 2/3. Consequently,
P(F |E1 )P(E1 )
1 · 1/3
=
= 1/2,
P(F )
2/3
P(E2 |F ) = 1 − P(E1 |F ) = 1/2.
P(E1 |F ) =
Let G be the event that the second child has brown eyes and notice that
P(G|E1 , F ) = 1
P(G|E2 , F ) = 1/2.
Then,
P(G|F ) = P(G ∩ E1 |F ) + P(G ∩ E2 |F )
= P(G|E1 , F )P(E1 |F ) + P(G|E2 , F )P(E2 |F )
= 1 · 1/2 + 1/2 · 1/2 = 3/4.
5