Download Theorem 4.4. Let E and F` be two events. Then In words, the

66
PROBABILITY IN FINITE SAMPLE SPACES / Chap. 2
each simple event. is nonnegative, it follows that the sum of the probabilities of the simple events of F is at least as large as the sum of the
probabilities of the simple events of E. But this is precisely the required conclusion.
In a Venn diagram, all the points representing elements of E would
also be in the region corresponding to F. To find peE) we add the
numbers on the flags erected at points of E. All of these numbers as
well as those, if any, that are erected at points in F but outside of E,
are summed to get P(F). Hence P(E) :$ P(F), as before.
Theorem 4.2 says that if event F occurs whenever event E occurs,
then the probability of F is at least as large as the probability of E.
Theorem 4.3. If E is any event, then 0
s P(E)
:$ 1.
Proof. We have E C S, since an event is by definition a subset of
the sample space S. Hence by Theorems 4.1 and 4.2, we conclude
that P(E) :$ peS) = 1. Also 0 C E, so that Theorem 4.2 yields
P(0) :$ P(E). Since P(0) = 0, our proof is complete.
The extreme values 0 and 1 are worthy of special attention. We
know that P(0) = 0 and peS) == 1. Recalling the definition of impossible and certain events given in the glossary on p. 52, we can
say that if an event is impossible, then it has probability 0, and if an
event is certain, then it has probability 1. But the converse of each
of these implications is false; i.e., if P(E) = 0 we cannot conclude that
E is impossible, and if P(E) = 1we cannot conclude that E is certain.
For example, in Solution 2 of Example 3.7, the event that the first
coin falls tails is {TH, TT}, certainly not the empty set. Yet, by
our assignment of probabilities to the simple events, this event has
probability O. In that same example, the event {HH, HT} has probability 1, but is not certain since it is not the entire sample space.
The reason for this state of affairs is that we have allowed simple
events to be assigned probability O. If we insisted, as some authors
do, that the probability of each simple event must be positive, then
only the empty event would have probability 0, and only the whole
sample space would have probability 1. However, it turns out to be
the case in problems involving infinite sample spaces that there must
exist events that are not impossible but yet have probability O. Although we cannot pursue this matter here, our definitions are formulated in such a way that the reader need not be surprised by this fact
when he goes on to studv »robabilities in infinite sample spaces.
Sec. 4 / SOME PROBABILITY THEOREMS
67
Theorem 4.4. Let E and F' be two events. Then
PCE U P) = PCE)
(4.2)
+ P(F)
- peE
n F).
In words, the probability that at least one of the events .E and F
occurs is obtained by adding the probability that E occurs and the
probability that F occurs, and then subtracting the probability that
both E and F occur.
Proof. First add the probabilities of all simple events containing
elements of E. Their sum is P(E). Then add the probabilities of all
simple events containing elements of F. Their sum is P(F). In the
sum peE)
PCF') we have included PC {OJ}) if and only if OJ e E U F
But we ha.ve added P( {oJ) twice for every OJ € En F': once in the
sum peE), and again in the sum PCP). The sum of the probabilities
of simple events tbat are counted twice is peE n F). We conclude
that peE) + P(F)·_· peE n F) is precisely the sum of the probabilities of all simple events in E U F', each counted once. Since this
sum is peE U P), the theorem is proved.
+
The reader should draw a Venn diagram and test his understanding
of this proof by formulating each step in the "numbers on flags" Ianguage. Before illustrating how Theorem 4.4 is used in a particular
example, we deduce two more results.
Theorem 4.5. If E and F are mutually exclusioe events, then
(4.3)
PCB U F)
=
PCB)
+ P(F).
Proof. In the result of Theorem ·1.4, we have only to note that
n F) = P(0) = O.
IlOW
PCE
Theorem 4.5 says that the probability of the occurrence of at least
one of two mutually exclusioe events is the sum of their individual
probabilities. Let us not forget the italicized hypothesis that must be
true before using Formula (4.3). The use of Formula (4.2) requires
no such caution, since if holds for any two events.
Theorem
(4.4)
4.6. Let E and E' be any complementary
peE')
=
events. Then
1 - PCE).
In words, the probability that E does not occur is obtained by sub..
tracting from 1 the probability that E does OCCllI'.
Proof. E and E' are mutually exclusive events, since EnE'
Hence by (4.3),
rt» U E') = PCE) + peE').
= 0.
68
But E U E'
4.1. Hence
PROBABILITY IN FINITE SAMPLE SPACES / Chap. 2
= S,
the entire sample space, and PCS) = 1by Theorem
Sec. 4 / SOME PROBABILITY THEOR£MS
69
~o to each simple event of S. Now E contains [~%Q.J* = 33 integers,
and is therefore
1 = peE)
+ peE'),
which is equivalent to (4.4).
In our less formal language, (4.4) is merely the result of noting that
we obtain the sum of the numbers on all flags in S by adding the sum
of the numbers on flags in E to the sum of the numbers on flags not
inE.
The following examples illustrate
compute probabilities.
how our formulas
can be used to
+
Example 4.1. Three coins are tossed. Find the probability of getting at least one head. We assign equal probabilities to the eight
simple events of the sample space S defined in (2.1), p. 51. If E is
the event "at least one head," then the complementary
event E' is
"no heads." By Theorem 4.6,
peE)
the union of 33 simple events, each with probability
-1030' Similarly,
F is the union of [~RQ.J = 25
simple events, so that P(F) = -ila. Since there are integers among
the first 200 (like 24 and 48) that are divisible by both 6 and 8, the
events E and F are not mutually exclusive. Hence we must compute
peE () F). An integer is divisible by both 6 and 8 if and only if it
is divisible by 24, the least common multiple of 6 and 8. There are
[~H] = 8 integers among the first 200 that are divisible by 24. Hence
peE () F) = 2-%-0. By applying Formula (4.2), we find the required
probability,
peE U F) = "2?o\
-iiJ%- - 2~O = t.
~k. Hence peE) =
= 1-
peE')
= 1-
P({TTT})
= 1-
i
=
i·
Note that we could have computed peE) directly by recognizing
that E is itself the union of seven simple events. This example is so
simple that either method is easy. But often in more complicated
problems, the most efficient way to find the probability of an event
is first to compute the probability of its complementary
event and
then use Formula (4.4). Recall that we followed this procedure in
solving the birthday problem. Although interested in the event E
(at least two people have the same birthday) we found it convenient
(see Example 2.1) to first study the event E' (no two people have the
same birthday).
of this result, see Problem 4.11.)
(For a generalization
We have seen that in many examples it is reasonable to assign the
same probability to each simple event of the sample space. In this
circumstance,
there is a simple formula for the probability of an
event.
Theorem 4.7. Suppose each of the n simple events of the sample
space S in (4.1) is assigned the same probability.
(This probability
must then be Iln.) If E is an event containing f elements, then
peE) =
(4.5)
L
n
In other words, the probability of an event is the ratio of the number
of elements in the event to the number of elements in the entire
sample space.
Proof. Since E contains f elements, E is the union of f simple events
of the sample space S. Hence, directly from the definition, peE) is
the sum of f probabilities, each equal to 1/n. But this sum is precisely fin, so that our proof is complete.
"favorable" to E whenever
as follows: If an experiment can result in ti equally likely outcomes, then the probability
of E is the ratio of the number of outcomes favorable to E to the total
number of outcomes. This is the classic "definition"
of probability
If we call an outcome of the experiment
E occurs, then Theorem 4.7 can be paraphrased
Example 4.2. An integer is chosen at random from the first 200
positive integers. What is the probability that the integer chosen is
divisible by 6 or by 8?
Let E be the event "integer selected is divisible by 6" and F the
event "integer selected is divisible by 8." Weare required to find
peE U F). We define S = {I, 2,3, "', 200} and assign probability
* The
symbol [;I:] stands for the greatest integer less than or equal to the number
z, Thus,
[3.6]
=
3,
m
=0,
[243] = 5,
etc.
70
PROBABILITY IN FINITE SAMPLE SPACES / Chap. 2
given by Laplace (1749-1827), one of the first and most important
contributors
to the mathematical
theory of probability.
Let us not
forget that this rule for computing probabilities is applicable only
when all simple events have been assigned the same probability.
Thus, Formula (4.5) does not apply to the wide variety of important
problems where it is not reasonable to make this special assignment
of probabilities to simple events.
Ordinarily, to compute peE) we must first determine which elements of the sample space are in E, and then we add the probabilities
of the corresponding simple events. But when Theorem 4.7 applies,
we need only know how many elements are in E. It is therefore extremely useful to have effective techniques for counting the elements
in sets specified by defining properties. In Example 3.6, for instance,
the probability of the event that at least two people have the same
birthday 'was easy to find because we had been able (in Example 2.1)
to count the elements in this event by using the fundamental
principle of counting. We discuss some other techniques for counting in
the next chapter. Until then, our examples will be chosen so as to
lead to events whose elements can be counted by explicit enumeration
or by use of the fundamental principle.
We conclude this section with a brief discussion of the relation between the probability of an event and "odds" for the event.
4.1. Let E be any event.
to b if and only if
We say that odds for E are a
Definition
peE)
=_.
a+b
If odds for E are a to b, then odds against E are b to a.
TABLE 11
Odds for
1 to 1
2 to 1
3 to 1
3to5
1 to 2
12 to 5
E
peE)
1.
Given the odds for E we have only to apply Definition 4.1 to find
the probability of E. On the other hand, if peE) is given, we write
it in the fractional form aj(a
b) and then know that the odds for
E are a to b. For example, if peE) = 0.7, we first write
+
peE)
= -7 = -_.7
10
7
+3
Hence, odds for E are 7 to 3. Since odds are often used to express
probabilities of events, it is useful to be able to translate odds to
probabilities and vice versa.
PROBLEMS
4.1. Two fair dice are rolled. Find the probability of the event T!.' that the
dots on the two uppermost faces do not add to 4. What are odds for E'?
4.2. A card is drawn at random from a standard deck of playing cards. Let
E be the event "card selected is an ace" and F the event "card selected
is a spade."
(a) Are E and F mutually exclusive events?
(b) Find the probability that at least one of the events E and 1'"occurs.
(c) What are odds for the event E U F?
4.8. A fair die is rolled twice. What are odds for the event that at least
one roll yields a number less than 3?
4.4. Odds a to band c to d are said to be equal if q:b = c:d, i.e., if their
ratios are equal. For example, odds of 10 to 5, 4 to 2, and 2 to 1 are
equal.
a
Table 11 gives some common odds and corresponding
71
Sec. 4 / SOME PROBABILITY THEOREMS
probabilities.
(a) Show that if odds for two events are equal, then the events have
equal probabilities.
(b) Show that odds against an event E are equal to odds for the complementary event E'.
4.5. Odds for event E are 2 to 1~ Odds for E U Fare 3 to 1. Consistent with
this information, what are the smallest and largest possible values for
the probability of event F?
2
i
.3
..
3
!
if
4.6. A card is drawn at random from an ordinary deck of 52 cards. This
card is replaced, and then another card is selected at random from the
full deck.
(a) Define a suitable sample space for this experiment and assign probabilities to its simple events.
72
PROBABILITY IN FINITE SAMPlE SPACES / Chap. 2
Sec ..• / SOME PROBABILITY THEOREMS
73
(b) Find the probability that at least one of the cards selected is the
ace of spades.
(c) What are the odds for the event that neither card is the ace of
spades?
4.14. Generalize Theorem 4.4 by showing that the probability of the occurrence of at least one among three events El, E2, and Ea is given by
4.7. Repeat the preceding problem, but now assume that the first card is
not replaced before the second is drawn.
[Note: You will find a Venn diagram like the one in Figure 7 helpful
in checking that the probability of each simple event making up
E, V E2 V Ea is counted once and only once in the expression on the
right in (4.6).]
4.8. The output of a machine producing nails is known to contain 2%
defectives, the other 98% meeting specifications. From the very large
lot of nails produced by the machine, two nails are drawn at random
and inspected.
(a) Define a suitable sample space for this experiment and make a
reasonable assignment of probabilities to its simple events.
(b) Find the probability that at least one of the nails is defective.
4.9. A high school senior applies for admission to college A and college B.
He estimates that the probability of being admitted to A is 0.7, that
his application will be rejected at B with probability 0.5, and that the
probability of at least one of his applications being rejected is 0.6.
What is the probability that he will be admitted to at least one of the
colleges?
4.10. If in Theorem 4.2 we make the hypothesis that E is a proper subset of P,
i.e., that E ~ P but E ;c P, does it then follow that P(E) < PcP)?
4.11. (a) An integer is chosen at random from the first 20 positive integers.
What is the probability that the integer chosen is divisible by 6
or 8?
(b) An integer is chosen at random from the first 2000 positive integers.
What is the probability that the integer chosen is divisible by 6
or 8?
(c) The result of Example 4.2 in the text together with the results of
parts (a) and (b) should lead you to conjecture a general theorem
of which these results are special cases. State such a theorem and
try to prove it.
4.12. Prove that if E and F are any events, then
peE r, F)
peE)
P(E V F)
peE)
s
s
s
+ P(F).
4.13. Let E and P be any two events. Suppose the numbers peE), PCP), and
peE n P) are known. Find formulas in terms of these numbers for the
following probabilities. In each case give a verbal description of the
event whose probability you are finding.
(a) P(E' V F')
(b) P(E' n P')
(c) P(E' V F)
(e) P(E
F')
r,
(d) P(E' r: F)
(f) P«E
F)')
r.
(4.6)
peEl
V E2 V Ea)
.- peEl
= P(EI)
n Ea) -
+ P(E2) + P(Ea) - peEl n E2)
P(E n Ea) + peEl n E2 n Ea).
2
4.16. From a standard deck we select one card at random. Use Formula
(4.6) to find the probability that the card is a spade, all honor card, or
a deuce.
4.16. Use Formula (4.6) to find the probability that a number selected at
random from the first 200 positive integers is divisible by 6 or 8 or 10.
4.17. We make a definition and then state a theorem. Use Formula (4.6) to
prove the theorem.
Definition 4.2 Let k be any integer greater than 1. Events EI, E2,
... , Ek are said to be mutually exclusive in pairs if and only if all
possible pairs of events from El, E2, ... , E. are mutually exclusive,
n
i.e., E,
B,
1,2, "., k.
=0
for all i ;c j where i and j can assume the values
Theorem 4.8 If EI, E2, and E3 are mutually exclusive in pairs, then
(4.7)
peEl
V
E2 V Ea) = peEl)
+ P(E2) + peEs).
n n
~U8. Suppose we assume only that E,
Ez
Ea = 0. Show by example
that (4.7) does not necessarily hold. (Cf. Problem I.4.6b.)
4.19. Prove the following generalization of Theorem 4.8 by mathematical
induction.
Theorem 4.9 Let k be any integer greater than 1 and suppose the
events EI, E2, ••• , Ek are mutually exclusive in pairs. Then
(4.8)
+ P(Ez) + ...
peEl V E2 V ". V Ek) = peEl)
-I-peEk).
4.20. Modify the hypothesis in Theorem 4.9 so that El, E2, ••• , Ek are any
events, not necessarily mutually exclusive in pairs. With this weaker
hypothesis, prove the following weaker result:
n»,
V Ez V ... V Ek)
::;
peEl)
+ P(E + ... + peEk).
2)
4.21. (a) Find the probability of at least one match when using a deck of
three cards. (Cf. Problem 3.9.)
(b) Find the probability of at least one match using four numbered
squares and four cards. (First define a sample space and make an
acceptable assignment of probabilities to its simple events.)
74
PROBABILITY IN FINITE SAMPLE SPACES / Chap. 2
(c) We want to find a formula for the probability of at least one
match when using N squares and a deck of N cards (numbered
1, 2, "', N), where N is any positive integer. Define a suitable
sample space for this experiment, determine the number of elements in this sample space, and then make an acceptable assignment of probabilities to its simple events. Note that we could
find the probability of at least one match if we had a formula for
P(EI U E2 U ... U EN), where E, denotes the event that a match
occurs at card number j. Can you guess this formula by detecting
a pattern in Formulas (4.2) and (4.6)? If not, then first use (4.2)
and (4.6) to derive a formula for the special case N = 4, and then
try guessing again. The proof of the correct general formula and
its use to find the probability of at least one match require counting
techniques that we have not yet discussed. * But even when we
can't complete a problem, it is useful to think about it and try
to see what we need to learn in order to be able to complete it.
This problem is the famous problem of rencontre in probability
theory and was originally discussed by the French mathematician
Montmort (1678-1719).
Sec. 5 / CONDITIONAL
Example 5.1. A club with five male and five female charter members elects two women and three men to membership. From the total
of 15 members, one person is selected at random. We are interested
in two events:
E = person selected is a male,
F = person selected is a charter member.
* See W. Feller, An Introduction to Probability Theory and Its Applications,
2nd edition, John Wiley and Sons, Inc., 1957, pp. 88-91. For another solution and
interesting historical comments, see I. Todhunter, A History of the Mathematical
Theory of Probability, Chelsea Publishing Co., 1949, pp. 91-93.
75
As sample space we take a set S of 15 elements, one for each club
member. Since the selection is "at random" we assign probability
1\ to each simple event of S. Observing that E is the union of eight
simple events, P the union of ten simple events, and E
F' the union
of five simple events, we calculate
n
P(E)
=
-l~5' P(F)
=H =
j,
peE n F)
=~
=
1.
So far we have nothing new. But now suppose we are informed
that the person selected is a charter member. What is the probability
of E, now that this fact about the outcome of the experiment has
been made known to us? Most people quickly answer that the revised probability of E should be rlr. They reason as follows: Since
F is known to have occurred, we know that one of the ten charter members was selected. The event E occurs if one of the five male charter
members is selected. Because the selection is at random, the probability of selecting one of the five males from the ten charter members
is -h-. If we introduce the symbol P(EIF') to denote this revised 01'
conditional probability of E given F, then
5. Conditional probability and compound experiments
Suppose an experiment is performed and we are interested in the
probability of some event E. But now assume that we are given additional information, namely, that another event F has occurred. In
this section, we discuss how the computation of the probability of E
is affected by the information that F is known to have occurred.
It is helpful first to take a close look at an example in which we can
find reasonable answers on intuitive grounds. The methods we employ in this simple example will lead us to formulate precise definitions that will become part of our mathematical theory.
PROBABILITY
P(E)
= -fr; and
P(EIF)
==
!.
Thus, in this example the probability of E decreases due to the added
information that event F has occurred.
This informal and intuitive reasoning can be described in another
way. Ordinarily, given a sample space S and an acceptable assignment of probabilities to the simple events of S, we compute the probability of an event E by adding the probabilities of the simple events
whose union is E. Since peS) = 1 and Ens
= E, we can write
the identity
(5.1)
P(E)
=
P(~(~~
82,
which shows that P(E) is the ratio of the probability of that part of
E included in S (which happens to be all of E) to the probability of
S itself (which happens to be 1).
But if we are told that event F has occurred, then the outcomes
corresponding to elements of F', the complement of F, are no longer
possible. Hence, in the light of our added information about. the outcome of the experiment, the event F replaces the sample space 8 as
the set. whose elements correspond to all possible outcomes of the