Download appendix - Cengage

APPENDIX
Introduction to Probability
Professor J. Anthony Capon
A Supplement to Healey, Statistics: A Tool for Social Research, 8th edition.
This supplement provides a more detailed and methodical treatment of probability than it was
possible to include in the text. It should be read in conjunction with Chapter 5. This material is
adapted from Elementary Statistics for the Social Sciences by Prof. J. Anthony Capon.
Statistical decision making – estimation (Chapter 7 of the text) and hypothesis testing (Chapters 8
to 11) – is based on probability theory and this supplement provides an introduction to this field.
People have been interested in probability and have played games of chance for thousands of years
but it was not until the 17th century that Blaise Pascal, Pierre deFermat, Christian Huygens, and
others outlined the beginnings of systematic study of probabilities – the mathematical science of
estimating the likelihood of a particular event or outcome such as roll of the dice or draw of a card.
In this supplement we shall examine some basic laws of probability. We will look at the
methods of calculating probabilities and combinations and permutations.
INTRODUCTION TO PROBABILITY
Probability may be defined as the frequency with which a particular event will occur in an infinite
number of trials. Probability is usually expressed as a proportion. If we let Y be the number of
trials, and X be the number of times a specified event E occurs, then the probability of event E
occurring may he expressed as
Formula 1 P[ E ] 
X
Y
An example will clarify how to calculate a probability. Suppose we have a new penny. If we flip
the coin, we will get a head (one event) or tail (another event). If we flipped the coin a very large
number of times, we would get H number of heads and T number of tails, assuming the coin cannot
land on its edge. We could express the probability of getting a head – P[H] – as
Formula 2 P[ H ] 
H
H T
If the coin always came up heads, the probability of getting a head would be 1.00. If it never came
up heads, the probability would be 0. Therefore, the maximum value a probability can be is 1; the
minimum is 0. But you do not have to be a statistician to know that if you flipped the coin a very
large number of times, you would get heads about half the time. Therefore, the probability of
getting a head would be about .5. Note that we require the coin to be flipped a very large number
of times, approaching infinity. We are not saying that if you flipped the coin 10 times, you would
get exactly 5 heads. What we are saying is that as the number of trials increases, the proportion of
heads will approach .5. Theoretically, if the number of trials is infinite, the proportion of heads will
be .5.
The type of probability we have been discussing so far is called a priori probability. It is
based on mathematical reasoning and logic. We cannot demonstrate that the probability of getting a
head when flipping a coin is .5, since by definition we could not flip the coin an infinite number of
times. But we can show that probability is .5 logically.
Another type of probability is called an empirical probability, which is determined by
observation and experimentation. Death rates, the probabilities of death in a given year for a given
age and sex, are based on empirically obtained mortality data. Insurance company statisticians use
these probabilities to calculate life insurance rates. While both types of probability are useful, the
type of decision making with which we shall be concerned in this text is based on a priori
probability.
CALCULATING PROBABILITIES
There are two basic rules used in the calculation of probabilities: the addition rule and the
multiplication rule. The addition rule deals with the probability of one of a number of possible
events occurring, usually in one trial. The multiplication rule deals with the probability of more
than one event occurring, usually in more than one trial.
The Addition Rule
When we apply the addition rule for probability, we want to know the probability of event A or
event B occurring. We may symbolize this probability as P[A or B]. In order to calculate P[A or B]
we first determine whether events A and B are mutually exclusive. Two events are mutually
exclusive if they cannot occur in the same trial. For example, if we flip a coin, we get either a head
(event A) or a tail (event B); we cannot get both in the same trial. Therefore, events A and B are
mutually exclusive in this case. If we draw one card from a deck of playing cards, we cannot get
both a spade and a heart. So these two events are also mutually exclusive. On the other hand, if we
drew one card, we could get both a spade and a 10 (the 10 of spades). These two events are not
mutually exclusive, since both could occur in one trial.
If two events are mutually exclusive, the probability of one or the other of them occurring in
the same trial is the sum of their respective probabilities. The addition rule – for mutually exclusive
events – can be stated as:
Formula 3
P[A or B] = P[A] + P[B]
We will illustrate the use of this formula with several examples. First, consider the probability of
getting a head or a tail in one flip of a coin. Using the addition rule:
P[H OR T] = P[H] + P[T]
= 0 5 + 0.5
= 1.00
You will always get a head or a tail when you flip a coin.
Second, suppose we have an urn in which there are eight red marbles, five blue marbles,
and three yellow marbles. If we draw one marble, what is the probability that it will be blue or
yellow? There are 16 marbles in the urn. The probability of drawing a blue marble (P[B]) is 5/16.
The probability of drawing a yellow marble (P[Y]) is 3/16. Therefore, since these events are
mutually exclusive, we can apply the addition rule:
P[B OR Y] = P[B] + P[Y]
= 5/16 + 3/16
= 8/16
= 0.5
As a third example, consider the probability for drawing a red king or a spade in a single draw from
a deck of cards. First, the two events are mutually exclusive, since there are no red kings that are
also spades. There are 52 cards in a deck, including 2 red kings and 13 spades. Therefore,
P[red K or S] = P[red K] + P[S]
= 2/52 + 13/52
= 15/52
= 0.29
Extending the Addition Rule to Two Events that are not mutually exclusive. Now suppose that
two events are not mutually exclusive. Then the probability of getting event X or event Y is the
sum of their respective probabilities minus the probability of both events occurring at the same
time. In notation,
Formula 4
P[A OR B] = P[A] + P[B] – P[A & B]
Where P[A&B] is the probability of events A and B occurring at the same time. We can illustrate
with an example. If we draw one card from a deck of cards, what is the probability that it will be a
club or a face card? We can calculate the probability by counting the cards. First, we have 13 clubs
(including the jack, queen, and king of clubs). We also have 9 other face cards (excluding the jack,
queen, and king of clubs). Therefore, since there are 22 cards in the deck that are either clubs [C]
or face cards [FC],
P[C OR FC] = 22/52 = 0.42
Now, let’s repeat the calculation using the formula. There are 13 clubs in the deck, 12 face cards
(jack, queen, and king of each of the four suits), and 3 clubs that are also face cards (jack, queen
and, king of clubs) Therefore,
P[C OR FC] = P[C] + P[FC] – P[C&FC]
= 13/52 + 12/52 – 3/52
= 22/52
= 0.42
Extending the Addition Rule to Three Mutually Exclusive Events. The addition rule may be
extended to three or more events. If we have three mutually exclusive events, probability of any
one of them occurring is simply the sum of their respective probabilities:
Formula 5
P[A OR B OR C] = P[A] + P[B] + P[C]
For example, if we draw one card from a deck of cards, what is the probability that it will be a 3, a
red 10, or a black face card? The three events are mutually exclusive. In the deck there are four 3s,
two red 10s, and six black face cards. Therefore,
P[3 or red 10 or black FC] = P[3] + P[red 10] + P[black FC]
= 4/52 + 2/52 + 6/52
= 12/52
= 0.23
We can extend the addition rule to any number of mutually exclusive events. In general, for any
number of mutually exclusive events, the probability of any one of them occurring is the sum of the
probabilities of each event.
Extending the Addition Rule to Three Events that are not Mutually Exclusive Let us now
consider the probability of more than two events that are not mutually exclusive. Extending the
addition rule to cover this circumstance is not nearly as straightforward as the case in which the
events are mutually exclusive. Since the events overlap with each other in three combinations (A
with B, A with C, B with C), we have to subtract the overlap of events three times. However, we
also have to take account of the overlap of all three events (A with B with C) and add it back into
the final calculation. So, to find the probability of three events which are not mutually exclusive,
we must use Formula 6:
Formula 6
P[A OR B OR C] = P[A] + P[B] + P[C] - P[A&B] - P[A&C] –
P[B&C] + P[A&B&C]
Let us consider an example. Suppose we draw one card from a deck. What is the probability
that it will be a red face card [FC], a king [K], or a diamond [D]? Note that these events are not
mutually exclusive since there is a red face card that is a king, red face cards that are also
diamonds, and a king that is also a diamond.
First, let us figure out the answer simply by counting cards. There are six red face cards –
the jacks, queens, and kings of hearts and diamonds. There are two kings we have not already
counted – the kings of spades and clubs. There are 10 diamonds we have not already counted – the
ace through 10 of diamonds. Therefore, there are 18 cards that fit our criteria, so the probability of
drawing any one of them must be 18/52 or 0.35.
Let us now use the formula:
P[red FC or K or D] = P[red FC] + P[K] + P[D] – P[red FC & K]
– P[red FC & D] – P[K & D] + P[red FC & K & D]
Where:
P[red FC] =
P[K] =
4
52
6
52
P[D] =
13
52
P[red FC & K] =
2
(kings of hearts and diamonds)
52
P[red FC & D] =
3
(jack, queen, and king of diamonds)
52
P[K & D] =
1
(king of diamonds)
52
P[red FC & K & D] =
1
(king of diamonds)
52
Therefore,
P [red FC or K or D] = 6/52 + 4/52 + 13/52 - 2/52 – 3/52 – 1/52 + 1/52
= 18/52
= 0.35
The addition rule may also be extended to four or more events that are not mutually
exclusive. However, the calculations become progressively more difficult, and we will not do so
here.
The Multiplication Rule
As we have seen, the addition rules is used when we want to determine the probability of event A
or event B occurring in one trial. The multiplication rule is used when we want to determine the
probability of event A followed by event B, in two trials. Just as with the addition rule, we have
two cases; one when the events are independent, and the other when the events are conditional.
Two events are independent if the occurrence of one event in no way affects the
probability of the second event occurring. For example, if we flip a coin twice, what is the
probability of getting a head followed by a tail? The probability of getting a head on the first flip is
1 out of 2 or 0.50. Regardless of what we get on the first flip, the probability of getting a tail on the
second flip is also 1 out of 2 or 0.50. So the two events are independent. Similarly, suppose we
roll a die twice. What is the probability of getting a 2 followed by a 4? What we get on the first
roll in no way affects what we get on the second roll. Both probabilities are 1 out of 6 or 0.17, and
the events are independent.
Now suppose we have a bowl of fruit containing 4 apples, 3 peaches, and 2 pears. If you
are going to eat two randomly selected pieces of fruit from this bowl, what is the probability that
they will be a peach followed by an apple? Since there are 9 pieces of fruit in the bowl, the
probability that the first fruit selected will be a peach is 3 out of 9 or 0.33. Once the peach is eaten,
however, there are only 8 pieces of fruit left, of which 4 are apples. The probability that the second
fruit will be an apple then is 4 out of 8, not 4 out of 9. So the probability of selecting an apple as
the second fruit is conditional upon the selection of the peach, since the first event (the selection of
the peach) affects the probability of the second event (the selection of the apple).
If two events are independent, the probability of the first event occurring followed by the
second event is the product of their respective probabilities. In notation, if we let [A&B] represent
A followed by event B, then:
Formula 7
P [A&B] =P [A] * P [B]
The probability of getting a head on the first flip of a coin followed by a tail on the second flip is:
P [H&T] =P [H] * P [T] = (0.50) * (0.50) = 0.25
The probability of getting a 2 followed by a 4 in two rolls of a die is:
P [2&4] = P [2] * P [4] = (0.17) (0.17) = 0.03
Now suppose we draw two cards from a deck of cards. What is the probability of getting a
face card followed by a 10? Are the two events independent or conditional? The answer depends
on what we do with the first card after we draw it. If we put it back in the deck, there will still be
52 cards in the deck, and the two events will be independent. If we do not put it back, however,
there will be only 51 cards remaining for the second draw, and the events will be conditional.
When we put the first card back, we call it drawing (sampling) with replacement. If we
do not put the first card back, we call it drawing (sampling) without replacement. Suppose we
draw with replacement, and thus have independent events. Since there are 12 face cards and 4 tens,
P [FC&10] = P [FC] * P [10]
= (12/52) * (4/52)
= (0.2308) * (0.0769)
= 0.0178
If two events are conditional, then the probability of the first event occurring followed by
the second event is obtained by multiplying the probability of the first event by the probability of
the second event given that the first event has occurred. If P [B/A] means “the probability of B
occurring given that A has occurred,” then:
P [A&B] = P [A] * P [B/A]
If we draw the two cards without replacement, the probability of the first card being a face card
remains 12/52. But since there will be only 51 cards remaining in the deck for the second draw, the
probability of the second card being a 10 is 4/51. Therefore,
P [FC&10] = P [FC] P * [10/FC]
= (12/52) * (4/51)
= (0.2308) * (0.0784)
= 0.0181
Let us return to the bowl of fruit, with 4 apples, 3 peaches, and 2 pears. Since we are
drawing without replacement, the probability of getting a peach followed by an apple is:
P [peach & apple] = P[peach] * P [apple/peach]
= (3/9) * (4/8)
= (0.33) * (0.50)
= 0.17
One final note on conditional probabilities:
P [A] P [B/A] = P [B] P [A/B]
In the previous example, this tells us that the probability of getting a peach followed by an apple is
the same as the probability of getting an apple followed by a peach:
P[peach] P [apple/peach] = P[apple] P [peach/apple]
(3/9)(4/8)= (4/9) (3/8)
12/72=12/72
Extending the Multiplication Rule to Three or More Events. Extending the multiplication rule
to three or more events is straightforward. If three or more events are independent, then the
probability of the events occurring in a particular order is the product of their representative
probabilities:
Formula 8 P [A&B&C] = P [A] * P [B] * P[C]
If an urn contains five red, four green, and two yellow marbles, then the probability of selecting a
red marble, followed by a yellow and a green in that order, with replacement, is:
P[R&Y&G] = P[R] P[Y] P [G]
= (5/11) * (2/11) * (4/11)
= (0.45) * (0.18) * (0.36)
= 0.03
If the events are conditional, then the probability of three events occurring in a particular order is:
Formula 9
P [A&B&C] = P [A] * P [B/A] * P[C/ (B&A)]
Where P[C/ (B&A)] is the probability of event C occurring given that events B and A have
occurred.
Suppose we draw the marbles without replacement in the first example of the section.
Then:
P[R&Y&G] = P[R] * P[Y/R] * P [G/(Y&R)]
After we draw the red marble, there will be 10 marbles remaining. After we then draw the yellow
marble, there will be 9 remaining. So:
P[R] P[Y/R] P [G/(Y&R)] = (5/11) * (2/10) * (4/9)
= (0.46) * (0.20) * (0.44)
= 0.04
The multiplication rules can be extended directly to any number of independent or conditional
events.
Combinations and Permutations
To this point we have been making an assumption that simplifies our calculations considerably.
With the multiplication rule, we have assumed that we want to calculate the probability of getting
event A followed by event B. In other words, we have assumed that the order in which the events
occurred was important. But in many circumstances we may be less interested in the order in
which events occurred and more interested in whether they occurred at all. For example, a bridge
player might be interested in the probability that his right-hand opponent was dealt both the king
and the queen of trumps. It is quite irrelevant whether the king was dealt before the queen or the
queen before the king.
When we are interested in the occurrence of a set of events regardless of order, we refer to
this as a combination. When we are interested in the order of a set of events, we refer to this as a
permutation.
Consider an example. Suppose we have three cards: a king (K), a queen (Q), and a jack (J).
If we are going to pick two cards at a time, and order is important, how many permutations can we
get? There are six:
KQ, QK, KJ, JK, QJ, JQ.
But if order is not important, then KQ is the same as QK, KJ the same as JK, and QJ the same as
JQ. So there are only three combinations:
KQ (or QK),
KJ (or JK),
QJ (or JQ)
It is often useful to know how many combinations or permutations of events can occur.
Fortunately, there are relatively simple formulae that give us the answers. If we have n things, and
we wish to take r of them at a time, then the number of combinations of n is given by
Formula 10
Cr 
n
n!
r !(n  r )!
The symbol n! (“n factorial”) means: n(n-1) (n-2)(n-3)…[n-(n-1)]
Thus, for example,
7!= 7 * 6 * 5 * 4 * 3 * 2 * 1 = 5,040
So if we have three cards (n), how many combinations (r) of two of them can we make?
n!
r !(n  r )!
3!
nCr 
2!(3  2)!
3  2 1
nCr 
2  1 1
6
nCr 
3
2
Cr 
n
As we saw with the three cards, there were indeed three combinations when we took the cards two
at a time.
Let’s consider another example. If we have seven persons, and are going to pick four of
them for a committee, how many choices do we have? Since the order in which we pick the
members of the committee is not important, we are asking how many combinations (r) of four
persons we can make from a group of seven (n):
n!
r !(n  r )!
7!
nCr 
4!(7  4)!
7  6  5  4  3  2 1
nCr 
4  3  2  1 3  2 1
5040
nCr 
 35
144
Cr 
n
There are 35 different committees of four persons that can he made from a group of seven.
When the order of things is important, we have a permutation, and the formula for the
number of permutations of n things taken r at a time is
Formula A11
Pr 
n
n!
(n  r )!
For the three cards in the previous example, the number of permutations of three cards (n)
taken two (r) at a time is
n!
(n  r )!
3!
3P2 
(3  2)!
3  2 1
3P2 
6
1
n
Pr 
And as we have already seen, there were six permutations of three cards taken two at a time.
If the order were important in the example where we were selecting four persons from
seven for a committee, we would have
n!
(n  r )!
7!
3P2 
(7  4)!
7  6  5  4  3  2 1
3P2 
 840
3  2 1
n
Pr 