Download Probability File

Probability
Probability 1
Classical Interpretation of Probability
Before we dive into the basic results and theorems of probability theory, it is worth considering
what we mean by “classical” probability theory. In essence, the classical interpretation of probability is
concerned with undertaking an “experiment” that has a number of possible outcomes, one of which
must occur as the outcome of the experiment. For example, when a die is rolled, the outcome of rolling
the die must be that either the number 1, 2, 3, 4, 5 or 6 is on the uppermost face of the die. In this case
the “experiment” is rolling the die, and the numbers 1 through to 6 form the possible outcomes of the
experiment.
At the heart of the classical interpretation of probability is the assumption that each of the possible
outcomes of an experiment is equally likely to occur. With this assumption, if there are N possible
outcomes of an experiment, then the probability P of any particular outcome O is given by
1
P (O ) =
N
Thus, the probability of throwing a die and getting a “1” is equal to 1/6, since there are six possible
outcomes, each of which is considered to be equally likely to occur.
Basic Definitions
In order to proceed, we need to make some basic definitions with respect to an experiment that we
undertake, and which leads to a number of possible outcomes.
Experiment: Experiment is an activity that is either observed or measured, such as tossing a coin, or
drawing a card.
The Sample Space – symbol S – denotes the set of all possible outcomes of the experiment. For
example, when a die is rolled and the uppermost number is noted, the sample space S consists of six
elements, written as follows:
S = {1,2,3,4,5,6}
An Event – symbol A – represents the subset of outcomes in S that satisfy some criterion for success in
the experiment. For example, if a particular event A is that the number obtained by rolling a die is even,
then the event A is
A = {2,4,6}
Since some events are impossible (e.g. rolling a “7” on a six-sided die), an event may contain no
elements at all. A set with no elements is referred to as an Empty Set.
Elementary Events: Elementary events are those types of events that cannot be broken into other
events. For example, suppose that the experiment is to roll a die. The elementary events for this
experiment are to roll a 1 or a 2, and so on, i.e., there are six elementary events (1, 2, 3, 4, 5, 6). Note
that rolling an even number is an event, but it is not an elementary event, because the even number can
be broken down further into events 2, 4, and 6.
Mutually Exclusive Events : Those events that cannot happen together are called mutually exclusive
events. For example, in the toss of a single coin, the events of heads and tails are mutually exclusive.
The probability of two mutually exclusive events occurring at the same time is zero.
Independent Events : Two or more events are called independent events when the occurrence or
nonoccurrence of one of the events does not affect the occurrence or nonoccurrence of the others. Thus,
when two events are independent, the probability of attaining the second event is the same regardless of
the outcome of the first event. For example, the probability of tossing a head is always 0.5, regardless of
Probability 2
what was tossed previously. Note that in these types of experiments, the events are independent if
sampling is done with replacement.
Collectively Exhaustive Events : A list of collectively exhaustive events contains all possible
elementary events for an experiment. For example, for the die-tossing experiment, the set of events
consists of 1, 2, 3, 4, 5, and 6. The set is collectively exhaustive because it includes all possible
outcomes. Thus, all sample spaces are collectively exhaustive.
Complementary Events (Ac): The complement of an event such as A consists of all events not
included in A. For example, if in rolling a die, event A is getting an odd number, the complement of A
is getting an even number. Thus, the complement of event A contains whatever portion of the sample
space that event A does not contain.
Equally likely events : When the events have equal chance to happen in an experiment , they are called
Equally likely events. Ex. In tossing a fair coin appearance of Head and Tail have equal chance.
Now consider two events A and B. Two important concepts are the union and intersection of A and B.
The Union of events A and B is the event that contains all the outcomes of A alone, all the outcomes
of B alone, and all the outcomes that belong to both A and B. The union of events A and B is written
as ( A ∪ B ) OR (A or B) and can be displayed diagrammatically as follows:
S
In this figure, the rectangle represents the total sample space S, and the ellipses
represent the subsets of S covered by the events A and B. The event that
represents the union of A and B is shaded blue in the figure.
The Intersection of events A and B is the event that contains all the outcomes
that belong to both A and B. The intersection of events A and B is written as
( A ∩ B ) OR (A and B) and can be represented diagrammatically as follows:
A
B
A
B
S
The event that represents the intersection of A and B is shaded blue in the figure.
Finally, the Complement of an event A – written AC – is the event that contains
all the elements of the sample space S that do not belong to A. The complement
of an event A can be represented diagrammatically as follows:
S
A
The event that represents the complement of A is shaded blue in the above figure.
As an example, consider the experiment of rolling a die, and let the event A be that the number rolled is
even, and let the event B be that the number rolled is three or less. Then:
A ∪ B = {1,2,3,4,6} ,
A ∩ B = {2},
AC = {1,3,5}
Mathematical Probability :
Consider a sample space that contains N outcomes, such that the probability of any one of the
outcomes is 1/N. Now consider an event A that is a subset of S and contains m outcomes. The
probability of the event A is given by
m
P ( A) =
N
As an example, consider rolling a die and let the event A be that the number rolled is even. Since there
are three outcomes in A (2,4 and 6) and six outcomes in S, it follows that m = 3 and N = 6, so that P(A)
= 3/6 = 1/2.
Now, two basic axioms of probability theory are as follows:
P ( A) ≥ 0
P (S ) = 1
Probability 3
P(Ac) = 1 – P(A)
Further important results are as follows:
P ( AC ) = 1 − P ( A)
P ( A ∪ B ) = P ( A) + P (B ) − P ( A ∩ B )
The last of these results can be visualised by considering the diagram above for the union of two
events. The probability is clearly related to the sum of the probabilities of the two events, but the
probability of the intersection of the two events needs to be subtracted out, since the intersection is
counted twice in the sum P(A) + P(B).
If the events A and B are independent of one another, then
P ( A ∩ B ) = P ( A )P (B )
Probability Laws:
1. When two or more events will happen at the same time, and the events are not mutually exclusive,
then:
P(X Y) = P(X) + P(Y) - P(X Y)
For example, what is the probability that a card chosen at random from a deck of cards will either be a
king or a heart?
P(King or Heart) = P(X or Y) = 4/52 + 13/52 - 1/52 = 30.77%
2. When two or more events will happen at the same time, and the events are mutually exclusive, then:
P(X Y) = P(X) + P(Y)
For example, suppose we have a machine that inserts a mixture of beans, broccoli, and other types of
vegetables into a plastic bag. Most of the bags contain the correct weight, but because of slight variation
in the size of the beans and other vegetables, a package might be slightly underweight or overweight. A
check of many packages in the past indicate that:
Weight.................Event............No. of Packages.........Probability
Underweight..........X.......................100.........................0.025
Correct Weight.......Y....................3600.........................0.9
Overweight............Z.......................300..........................0.075
Total..............................................4000..........................1.00
What is the probability of selecting a package at random and having the package be under weight or
over weight? Since the events are mutually exclusive, a package cannot be underweight and overweight
at the same time. The answer is: P(X or Z) = P(0.025 + 0.075) = 0.1
3. When two or more events will happen at the same time, and the events are dependent, then the
general rule of multiplication law is used to find the joint probability:
P(X Y) = P(X) . P(Y|X)
For example : What is the probability of selecting one card at random from a deck of cards and finding
the probability that the card is an 8 and a diamond. P(8 and diamond) = (4/52) . (1/4) = 1/52 which is =
P(diamond and 8) = (13/52) . (1/13) = 1/52.
4. When two or more events will happen at the same time, and the events are independent, then the
special rule of multiplication law is used to find the joint probability:
Probability 4
P(X Y) = P(X) . P(Y)
If two coins are tossed, what is the probability of getting a tail on the first coin and a tail on the second
coin?
P(T T) = (1/2) . (1/2) = 1/4 = 25%.
This can be shown by listing all of the possible outcomes: T T, or T H, or H T, or H H.
Conditional Probability
In many situations, an experiment may consist of a number of components that can be considered to be
independent. For example, consider an experiment in which two dice are rolled. If the two throws of
the dice are random, then the probability that the second die lands as a six (or any other number) is
completely independent of the outcomes of the throw of the first die, and is equal to 1/6.
However, consider now two events A and B. Suppose that event B occurs first. If A and B are
independent, then the probability that A occurs does not depend on the probability that B occurs.
However, there may be some circumstances in which the probability that A occurs is contingent on
whether the event B has occurred or not. That is, if we know that the event B has occurred, then that
knowledge influences the probability that A occurs.
In this case, we talk of the conditional probability that A occurs, given that event B has occurred. This
conditional probability is written as follows:
P( A B)
and is usually termed the “conditional probability of A given B”.
The conditional probability of A given B is evaluated from the following expression:
P( A ∩ B)
P(A B) =
P (B )
The plausibility of this formula can be confirmed by rewriting it slightly as
P ( A ∩ B ) = P ( A B )P (B )
This makes the not unreasonable assertion that the probability of A and B occurring is equal to the
probability of B occurring, multiplied by the probability of A given that B has occurred. One can also
write this in the form
P ( A ∩ B ) = P (B A)P ( A)
This result follows by swapping A and B around in expression (A). Note that if A and B are
independent then
P ( A ∩ B ) = P ( A)P (B )
P ( A B ) = P ( A)
Similarly, if A and B are mutually exclusive:
P( A ∩ B) = 0
P( A B) = 0
Example 1:
Consider the rolling of two dice. Compute the probability that (a) the sum of the rolled numbers is 7
(b) both dice show the same number (c) both dice show an even number.
In this case, the sample space S contains 36 outcomes. This is because the first die can give six values,
as can the second die. The sample space is therefore composed of the following 36 pairs of possible
outcomes:
S = (11, 12, 13, , 63, 64, 65, 66)
To evaluate the required probabilities, we define the appropriate event A and hence work out how many
of the 36 outcomes are present in A.
Probability 5
(a) Let the event A be the set of outcomes for which the sum of the numbers is 7. The event A is equal
to
A = {16, 25, 34, 43, 52, 61}
and hence contains 6 elements. The required probability is therefore 6/36 = 1/6.
(b) Let the event A be the set of outcomes for which the numbers on the dice are the same. In this case,
the event A is equal to
A = {11, 22, 33, 44, 55, 66}
and so the probability of A is once again 6/36 = 1/6.
(c) We could once again define an event A and count all the possible outcomes for which both numbers
are even. However, an alternative way to solve the problem is to note that the two throws of the dice
are independent. Hence we can define events A and B to be equal to the outcomes that the first die
gives an even number, and the second die gives an even number, respectively. Hence P(A) = 1/2 and
P(B) = 1/2. Thus the required probability is
P ( A ∩ B ) = P ( A )P (B ) = 1/ 4
Example2 (Conditional Prob.): Two dice are rolled and the sum of the two numbers is odd. What is
the probability that the sum is less than 8? (This example is taken from the book Probability and
Statistics, by Morris DeGroot, 1989).
We begin by letting A be the event that the sum is less than 8, and B be the event that the sum is odd.
Now, in this case,
A = {2,3,4,5,6,7,8}
B = {3,5,7,9,11}
A ∩ B = {3,5,7}
Now, there are two ways in which the sum can be 3 or 11, four ways in which the sum can be 5 or 9,
and six ways in which the sum can be 7. Therefore,
P (B ) = 1 / 2
P ( A ∩ B ) = 1/ 3
Therefore the probability that we seek, namely the probability that the sum is less than 8 given that the
sum is odd, is equal to
P ( A ∩ B ) 1/ 3 2
=
=
P(A B) =
1/ 2 3
P (B )