Download Lecture 17

2 Interpretations of Probability
In addition to the many formal applications of probability theory, the concept of probability
enters our everyday life and conversations.
 It probably will rain tomorrow.
 It is very likely that the bus will arrive late.
 The chances are good he will win the game.
Despite the fact that the concept of probability is such a common and natural part of our
experience, no single scientific interpretation of the term probability is accepted by all
statisticians.
Frequency interpretation: In many problems, the probability that some specific outcome of a
process will be obtained can be interpreted to mean the relative frequency with which that
outcome would be obtained if the process were repeated a large number of times under
similar conditions.
Example: the probability of obtaining a head when a coin is tossed is approximately 1/2.
Difficulties: Large number, similar conditions.
The frequency interpretation applies only to a problem in which there can be, at least in
principle, a large number of similar repetitions of a certain process. Many important
problems are not of this type. For example, the probability that a particular medical
research project will lead to the development of a new treatment for a certain disease
within a specified period of time.
Remark: The mathematical theory of probability can be usefully applied, regardless of which
interpretation of probability is used in a particular problem. The theories and techniques that
will be presented in this book has served as valuable guides and tools in almost all
aspects of the design and analysis of effective experimentation.
2 Experiments and Events
The term “experiment” is used in probability theory to describe virtually every process
whose outcome is not known in advance with certainty. Here are some examples.
(1) In an experiment in which a coin is to be tossed 10 times, the experimenter might
want to determine the probability that at least four heads will be obtained.
(2) In an experiment in which the air temperature at a certain location is to be observed
every day at noon for 90 successive days, a person might want to determine the
probability that the average temperature during this period will be less than some specified
value.
(3) In evaluating an industrial research and development project at a certain time, a
person might want to determine the probability that the project will result in the successful
development of a new product within a specified number of months.
The interesting feature of an experiment is that each of its possible outcomes can be
specified before the experiment is performed, and probabilities can be assigned to various
combinations of outcomes that are of interest.
3 Set Theory
3.1 The sample space
The collection of all possible outcomes of an experiment is called the sample space of the
experiment. Each possible outcome is called a point, or an element, of the sample space.
The sample space can be classified into two types: countable and uncountable. A sample
space is countable if there is a one-to-one correspondence between the elements of A
and the set of natural numbers {1, 2, 3, . . .}. Otherwise, it is uncountable.
3.2 Relations of set theory
Let S denote the sample space of some experiment. Then each possible outcome s of
the experiment is said to be a member of the space S, or to belong to the space S.
The statement that s is a member of S is denoted symbolically by the relation s 2 S.
An event C is a certain subset of possible outcomes in the space S. We say the event
C occurs if the outcome of the experiment is in the set C.
Example 3.1 A six-sided die is rolled. The sample space is S
Let A = {2,
4, 6} denote the event A that an even number is obtained, and B = {4, 5, 6} denote the
event that a number greater than 3 is obtained. We say A occurs if we get the number
2 in a roll.
= {1, 2, 3, 4, 5, 6}.
It is said that an event A is contained in another event B i f every outcome that belongs
to the subset defining the event A also belongs to the subset defining the event B. The
relation is denoted by A ½ B. Mathematically, we have
A ½ B , x 2 A ) x 2 B
(containment)
If two events are so related that A ½ B a nd B ½ A , then A = B, which means
that A and B
m ust contain exactly the same sample points. Mathematically, we have
A = B , A ½ B and B ½ A (equality)
A subset of S that contains no outcomes is called the empty set, or null set, and it is
denoted by the symbol ;. For each event, it is true that ; ½ A ½ S .
3.3 Operations of set theory
Given any two events (or sets) A a nd B, we have the following elementary set
operations:
Union: The union of A a nd B, written A [ B , is the set of elements that belong to
either A or B or both:
A [ B = {x : x 2 A or x 2 B}
Intersection: The intersection of A and B, written A \ B, is the set of elements that
belong to both A and B:
A \ B = {x : x 2 A and x 2 B}
Complements: The complement of A, written A c , is the set of all elements that are not in A:
Ac = {x : x
 A}
Disjoint events: It is said that two events A and B are disjoint, or mutually exclusive, if
A and B have no outcomes in common. It follows that A and B are disjoint if and only
if A \ B = ; .
The events A1, . . . ,An or the events A1,A2, . . . are disjoint if for every i 6= j, we have
that Ai and Aj are disjoint, that is, Ai \ Aj = ; for all i 6= j.
Example 3.2 (Event operations) Consider the experiment of selecting a card at random
from a standard deck and noting its suit: clubs (C), diamond (D), hearts (H), or
spades (S). The sample space is
and some possible events are
S = {C,D,H, S},
A = {C,D}, and B = {D,H, S}.
From these events we can form
A [ B = {C,D,H, S}, A \ B = {D}, and Ac = {H, S}.
Furthermore, notice that A [ B = S and ( A [ B ) c = ;.
4 The Definition of Probability
4.1 Axioms and Basic Theorems
In order to satisfy the mathematical definition of probability, P(A), the probability assigned
to event A, must satisfy three specific axioms.
Axiom 1 For every event A, P(A) ¸ 0. In words, the probability of every event must be
nonnegative.
Axiom 2 P(S) = 1. If an event is certain to occur, then the probability of that event is
1.
Axiom 3 For every infinite sequence of disjoint events A1,A2, . . .,
The mathematical definition of probability can now be given as follows: A probability
distribution, or simply a probability, on a sample space S is a specification of numbers
P(A) that satisfy Axioms 1, 2, and 3.
Theorem 4.1 P(;) = 0.
Theorem 4.2 For every finite sequence of n disjoint events A1, . . . ,An,
Theorem 4.3 For every event A, P(Ac) = 1 − P (A).
Theorem 4.4 If A ½ B, then P(A) ≤ P (B).
Theorem 4.5 For any event A, 0 ≤ P (A) ≤ 1 .
Theorem 4.6 For every two events A and B,
P(A [ B) = P(A) + P(B) − P(AB).
5 Finite sample spaces
5.1 Requirements of probabilities
In an experiment of which the sample space contains only a finite number of points s1, . .
. , s n, a probability distribution on S is specified by assigning a probability pi to each point
si 2 S. In order to satisfy the axioms for a probability distribution, the numbers p1, . . . , pn
must satisfy the following two conditions:
pi ¸ 0 , for i = 1, . . . , n
and
The probability of each even A c an then be found by adding the probabilities pi of all
outcomes si that belong to A.
Example 5.1 Consider an experiment in which five fibers having different lengths are
subject to a testing process to learn which fiber will break first. Suppose that the lengths
of the five fibers are 1,2,3,4, and 5 inches, respectively. Suppose also that the probability
that any given fiber will be the first to break is proportional to the length of that fiber.
A sample space S c ontaining n o utcomes s1, . . . , sn is called a simple sample space if
the probability assigned to each of the outcomes s1, . . . , sn is 1/n. If an event A i n this
simple sample space contains exactly m o utcomes, then
5 Finite sample spaces
Example 5.2 (Tossing coins) Suppose that three fair coins are tossed simultaneously. We
shall determine the probability of obtaining exactly two heads.
6 Counting Methods
In many experiments, the number of outcomes in S i s so large that a complete listing of
these outcomes is too expensive, too slow, or too likely to be incorrect to be useful. In
such an experiment, it is convenient to have a method of determining the total number of
outcomes in the space S a nd in various events in S w ithout compiling a list of all these
outcomes.
Multiplication rule
For example, suppose that there are three different routes from city A to city B and five
different routes from city B t o city C. Then the number of different routes from A to C
that pass through B is 3 × 5 = 15.
6.2 Permutations
Suppose that k c ards are to be selected one at a time and removed from a deck of n
c ards (k = 1, 2, . . . , n). Then each possible outcome of this experiment will be a
permutation of k cards from the deck, and the total number of these permutations will be
Pn,k = n(n − 1) · · · (n − k + 1). This number Pn,k is called the number of permutations of n
e lements taken k a t a time.
If k = n, Pn , n = n(n − 1) · · · 1 = n!. Note that 0! = 1.
Example 6.1 (Arranging books) Suppose that six different books are to be arranged on a
shelf. The number of possible permutations of the books is 6! = 720.
Example 6.2 (Ordered sampling without replacement) Suppose that a club consists of 25
members, and that a manager and a secretary are to be chosen from the membership.
The number of all possible choices is
P25,2 = 25 × 24 = 600.
Example 6.3 (Ordered sampling with replacement) Consider the following experiment: A
box contains n balls numbered 1, . . . , n. First, one ball is selected ar random from
the box and its number is noted. Th.is ball is then put back in the box and another ball
is selected. As many balls can be selected in this way. This process is called ordered
sampling with replacement. The sample space of this experiment will contain all vectors of
the form (x1, . . . , xk), where xi is the outcome of the ith selection. The total number
of vectors in S is nk. The probability of the event that each of the k balls that are
selected will have a different number is
Example 6.4 (The birthday problem) The problem is to determine the probability p that at
least two people in a group of k people (2 ≤ k ≤ 365) will have the same birthday.
Under appropriate assumptions (independence and uniformity),the probability that all k
persons will have different birthdays is P365,k / 3 65k. The probability that at least two of the
people will have the same birthday is
7 Combinatorial Methods
7.1 Combinations
Suppose that there is a set of n d istinct elements from which it is desired to choose a
subset containing k e lements (1 ≤ k ≤ n ). The number of different subsets that can be
chosen is
The number Cn,k is also denoted by the symbol
. When this notation is used, this
number is called a binomial coefficient because it appears in the binomial theorem.
Binomial theorem. For all numbers x and y and each positive integer n,
Examples
Example 7.1 (Unordered sampling without replacement) Suppose that a class contains 15
boys and 30 girls, and that 10 students are to be selected for a special assignment. We
shall determine the probability that exactly 3 boys will be selected.
The desired probability is
In summary, we have the following table for counting.
Example 7.3 (Unordered sampling with replacement) A restaurant has n items on its
menu. During a particular day, k customers will arrive and each one will choose one item.
The manger wants to count how many different collections of customer choices are possible
without regard to the order in which the choices are made.