Download An Introduction to Probability

An Introduction to Probability
(a monograph)
機率學
蔡文祥
講座教授
Wen-Hsiang Tsai, Ph. D.
Chair Professor, NCTU
Department of Computer Science
National Chiao Tung University
February 2012
Chapter 1
Combinatorial Analysis
1.0 Introduction to Probability Theory
 Why study the probability theory? -- Probability is a common sense for scholars and people in modern days.
 No engineer or scientist can conduct research and development works without
knowing the probability theory.
 Academic fields based on the probability theory --A lot of academic fields are based on the probability theory:
 statistics
 communication theory
 computer performance evaluation
 signal and image processing
 pattern recognition
 queuing theory
 stochastic processes
 quality control
 game theory ....
 Applications of the probability theory --A lot of applications are based on the probability theory:
 character recognition
 speech recognition
 opinion survey
 missile control
 weather prediction
 seismic analysis ...
 Course requirement and others -- Prerequisite course: calculus.
 Grade evaluation: home works 25%; two midterm exams + one final, each
25%.
 Office hours: right before and after class at the classroom or Computer Vision
Lab (EC129).
1- 1
 Reference textbook --Sheldon Ross, A First Course in Probability, 8th Edition, Prentice-Hall
International Inc., NJ, U. S. A., 2009.
 Some interesting questions --Do you have answers for the following questions?
 Is there a strategy to win in playing a slot machine in a casino?
 Do you know the meanings of people’s opinion polls reported in daily
newspapers?
With the election fast approaching, the Taiwan Thinktank has released its latest opinion
poll figures. KMT presidential candidate Ma Ying-jeou has a support rate of 39.5 percent,
with DPP candidate Tsai Ing-wen a close second, at 39.1 percent, while PFP candidate
James Soong lags at 11.1 percent. The polls were conducted on Dec. 23 and 24, canvassing
opinion from 1,067 adults above the age of 20. The confidence level of the poll is 95 percent
and margin of error 3 percent. (2011/12/26)
(大選腳步逼近，台灣智庫選情分析，公佈剛出爐的民調，馬英九拿到百分之 39.5 的支持度，
蔡英文也有百分之 39.1，宋楚瑜得到百分之 11.1，這份在 12/23、24 日進行的民調，電話抽查全
國 20 歲以上民眾，完成 1067 份樣本，信心水準 95%抽樣誤差 3%。)
 Are there really pre-known winning numbers (明牌) for lotto games?
1- 2
1.1 Introduction to Combinatorial Analysis
 Combinatorial analysis is a mathematical theory of counting.
 Many problems in probability can be solved by simply counting the number of
different ways in which a certain event can occur.
1.2 Principle of Counting
 Some terms -- experiment: any human activity, like tossing a die (骰子).
 outcome: a result of an experiment, like 5 points on a side of a die.
 Basic product principle of counting --“If experiment 1 has m possible outcomes and if for each outcome of
experiment 1, experiment 2 has n possible outcomes, then an experiment of
performing experiment 1 and experiment 2 simultaneously, called experiment 1 &
2, has mn possible outcomes.”
Proof: Easy; omitted.
 Example 1.1 --In tossing simultaneously a coin with two sides A and B and a die with six
sides 1 through 6, how many possible outcomes will appear?
Solution:
 The tossing may be regarded as an experiment (experiment 1 & 2) consisting
of two smaller experiments, tossing a coin (experiment 1) followed by tossing
a die (experiment 2).
 Experiment 1 has 2 outcomes.
 Experiment 2 has 6 outcomes.
 So experiment 1 & 2 has 26 = 12 outcomes.
 Generalized product principle of counting --“If Experiments 1 through k have n1 through nk outcomes, respectively, then
the experiment 1 & 2 & … & k has n1n2…nk outcomes.”
Proof: Easy to derive from the basic principle by induction.
 Basic sum principle of counting --“If Experiment 1 has m possible outcomes and if experiment 2 has n possible
outcomes, then an experiment which might be experiment 1 or experiment 2,
called experiment 1 or 2, has m + n possible outcomes.”
Proof: Easy; omitted.
 Example 1.2 --In tossing an object which might be a coin (with two sides A and B) or a die
(with six sides 1 through 6), how many possible outcomes will appear?
Solution:
1- 3
 The tossing may be tossing a coin (experiment 1) or tossing a die (experiment
2), or just experiment 1 or 2.
 So the number of outcomes is 2 + 6 = 8 according to the above basic sum
principle of counting.
 Generalized sum principle of counting --“If Experiments 1 through k have respectively n1 through nk outcomes, then the
experiment 1 or 2 or … or k has n1 + n2 + … + nk outcomes.
Proof:
Easy to derive from the basic principle by induction.
1.3 Permutations
 Fact 1.1: permutations of n objects --The number of different permutations of n different objects is
n!  n(n  1)(n  2)…21.
where “” means “equal to by definition.” (Note: “n!” is pronounced as “n
factorial.”).
Proof:
 The operation of finding all permutations may be regarded as an integrated
experiment of filling n objects into n slots as follows.
 Experiment 1: filling an object into the 1st slot with n choices (outcomes).
 Experiment 2: filling an object into the 2nd slot with n  1 choices
(outcomes).
…
 Experiment n: filling an object into the nth slot with 1 choice (outcome).
 So, according to the generalized product principle of counting, this integrated
experiment has the following number of outcomes (i.e., the following number
of permutations):
n(n  1)(n  2)…21 = n!.
 Fact 1.2: permutations of r objects selected from n ones --The number of different permutations of r objects selected from n different
objects is
P(n, r) 
n!
.
(n  r )!
Proof:
 The operation of finding all permutations of r objects selected from n different
objects may be regarded as an integrated experiment of filling r objects into n
slots as follows.
 Experiment 1: filling an object into the 1st slot with n choices (outcomes).
1- 4
2: filling an object into the 2nd slot with n  1 choices
(outcomes).
 Experiment
…
r: filling an object into the rth slot with n  (r  1) choices
(outcomes).
 So, the use of the generalized product principle of counting leads to the
following number of permutations:
 Experiment
n(n  1)(n  2)…[n  (r  1)]
= [n(n  1)(n  2)…21] / [(n  r)(n  r 1)…21]
= (n!)/(n  r)!
Done.
 Example 1.3 --In permuting the six letters in the word “avatar” (a title used for the movie “阿
凡達”), how many possible outcomes will appear?
Solution:
 Let the three a’s in the word be distinguished as a1, a2 and a3, respectively.
Then all the six letters are different, so the number of permutations of them
(called labeled permutations) is n! = 6!.
 However, consider each of the real permutations without distinguishing the
three a’s --- e.g., w = ratava. (Note: the Latin word e.g. means “for example.”)
 The following are all of the 6 (=3!) labeled permutations among the 6! ones,
which come from permuting the three labeled a’s in w = ratava:
ra1ta2va3, ra1ta3va2, ra2ta1va3,
ra2ta3va1, ra3ta1va2, ra3ta2va1.
 All these six labeled permutations should be considered as an identical real
permutation, which is w = ratava.
 Since each real permutation has six of such labeled permutations coming from
the three a’s, we conclude that the desired number of real permutations is just
6!/3! = 654 = 120.
 Fact 1.3: permutations of indistinguishable objects --The number of different permutations of n objects with n1 alike, n2 alike, ..., nr
alike and n = n1 + n2 + ... + nr is
n!
.
n1 !n2 !... nr !
(Note: alike means the objects in a group are indistinguishable from one another.)
Proof:
 By the same reasoning used in Example 1.3, we know that the number of
labeled permutations is n!.
1- 5
 Also, according to the generalized product principle of counting, each real
permutation has n1!n2!...nr! corresponding labeled permutations because there
are n1 objects alike, n2 objects alike, …, and nr objects alike.
 Therefore, the number of real permutations is (n!)/(n1!n2!...nr!).
1.4 Combinations
 Fact 1.4: basic formula for combinations --The number of different groups of r items that could be formed from a set of n
distinct objects with the order of selections being ignored is
n
n!
C(n, r)    
.
 r  (n  r )!r !
(Note: the objects selected to be in a group are regarded as indistinguishable.)
Proof:
 Since in the formation of each desired r-item group, the n objects are divided
into two groups (one with r items and the other with n  r items), with all the
objects in either group being regarded as indistinguishable.
 So, the problem may be considered as one of finding permutations from n
objects with r ones alike and the remaining (n  r) ones also alike (because
objects in a group are considered as indistinguishable).
 Therefore, according to Fact 1.3 above, the desired number is just
n!
.
(n  r )!r !
 Example 1.4 --From a group of 5 men and 7 women, how many different committees
consisting of 2 men and 3 women can be formed?
Solution:
 Experiment 1: select 2 men from 5.
 Experiment 2: select 3 women from 7.
 Experiment of forming a committee: experiment 1 & 2.
 #possible outcomes of experiment 1 = C(5, 2) = (54)/(21) = 10.
(Note: we use # to mean “number of.”)
 #possible outcomes of experiment 2 = C(7, 3) = (765)/(321) = 35.
 #possible outcomes of experiment 1 & 2 = C(5, 2)C(7, 3) = 350 by the basic
product principle of counting
 That is, the desired number of possible outcomes of the experiment of forming
a committee is 350.
 Fact 1.5: two identities about computations of combinations --The following two identities are true:
1- 6
C(n, r) = C(n, n  r);
C(n, r) = C(n  1, r  1) + C(n  1, r).
Proof: as exercises.
 Binomial Theorem --The following identity is true:
(x + y)n =
n
 C (n, k )x k y nk .
k 0
Proof:
 The proof can be done by combinatorial analysis.
 Consider (x + y)n as a product of labeled x’s and y’s:
(x + y)n = (x1 + y1)(x2 + y2)…(xn + yn).
 For example, we have
(x1 + y1)(x2 + y2)(x3 + y3) = (x1x2 + x1y2 + y1x2 + y1y2)(x3 + y3)
= x1x2x3 + x1y2x3 + y1x2x3 + y1y2x3 + x1x2y3 + x1y2y3 + y1x2y3 + y1y2y3
= x3 + x2y + x2y + x2y + … + yyy (with no label)
 There are in general 2n terms (like x1x2x3) in the result (for the above example,





there are 23 = 8 terms).
In each term (like x1x2y3), there are n factors (like x1, x2, y3 in x1x2y3) and there
is one factor for each i = 1, 2, …, n (like x2 for i = 2).
Among the 2n terms, how many of them will contain k of the x’s and n  k of
the y’s (k 個 x 以及 n  k 個 y)?
Since there are n factors in each term for selection of k x’s, the answer is C(n,
k) according to Fact 1.4 (basic formula for combinations).
That is, we have C(n, k) terms of the form xkynk, or equivalently, we have the
item C(n, k)xky n  k in the result, where k may take the values of 0 through n.
Therefore, by the basic sum principle of counting, the final result is
(x + y)n =
n
 C (n, k )x k y nk .
k 0
1.5 Multinomial Coefficients
 Fact 1.6: divisions of n objects into r groups --The number of different ways of dividing a set of n distinct objects into r
distinct groups of respective sizes n1, n2, ..., nr is
C(n; n1, n2, …, nr) 
Proof:
1- 7
n!
.
n1! n2 !... nr !
 The total operation can be divided into r steps, one following another,





meaning that the generalized product principle of counting is applicable.
First, choose n1 objects from the n ones, and the number of ways to do this is
N1 = n!/n1!(n  n1)!.
Then, choose n2 objects from the remaining n  n1 ones, and the number of
ways to do this is N2 = (n  n1)!/n2!(n  n1  n2)!.
…
Finally, choose nr objects from the remaining n  n1  n2  …  nr1 = nr ones
and the number of ways to do this is Nr = (n  n1  n2  …  nr1)!/nr!(n  n1 
n2  …  nr1  nr)! = (n  n1  n2  …  nr1)!/nr!0!. (Note: 0! = 1 by
definition.)
Then, by applying the generalized product principle of counting, we get the
following desired number of divisions after some reductions:
N1N2…Nr
= [n!/n1!(n  n1)!][(n  n1)!/n2!(n  n1  n2)!]…
[(n  n1  n2  …  nr1)!/nr!(n  n1  n2  …  nr1  nr)!]
n!
=
 C(n; n1, n2, …, nr).
n1 ! n2 !...nr !
 ***Multinomial theorem --(*** means “may be skipped”)
The following identity is true:
(x1 + x2 + … + xr)n =

C (n; n1 , n2 ,..., nr )x1n1 x2n2 ...xrnr
( n1 , n2 ,..., nr ):
n1  n2 ... nr  n
where the notation (n1, n2, …, nr): n1+n2+…+nr = n under the summation symbol
 means that the sum is performed over all possible nonnegative integer-valued
vectors (n1, n2, …, nr) such that n1 + n2 + … + nr = n.
Proof: as an exercise.
1.6 On Distribution of Balls in Urns
 Fact 1.7: ways of distributing n distinguishable balls into r urns --The number of ways for n distinguishable balls to be all distributed into r urns
is:
r  r  ...  r  r n .
n times
Proof:
 Each ball has r urns to choose.
1- 8
 The operation may be regarded as n experiments in a sequence.
 So, by the generalized product principle of counting, we get the desired
number of ways to be
r  r  ...  r  r n .
n times
 Proposition 1.1 --The number of positive integer-valued vectors (x1, x2, ..., xr) such that x1 + x2
+ ... + xr = n is C(n  1, r  1).
Proof.
 The problem is equivalent to choosing r  1 spaces in n  1 ones as illustrated
in the following diagram so that the #objects between every two neighboring
chosen spaces (specified by the symbol ^) becomes nonzero:
o^o o^o o ... o.
n objects
 So, the number of ways for doing this is C(n  1, r  1).
 Proposition 1.2 --The number of nonnegative integer-valued vectors (x1, x2, ..., xr) such that x1 +
x2 + ... + xr = n is C(n + r  1, r  1).
Proof.
 To meet the requirement of “nonnegativeness,” let yi = xi + 1 so that yi
becomes positive and y1 + y2 + ... + yr = n + r.
 Then, we can use Proposition 1.1 to derive the desired result, namely, C(n + r
 1, r  1).
 Fact 1.8: ways of distributing n indistinguishable balls into r urns --The number of ways for n identical balls to be distributed into r urns is just
the number of nonnegative integer-valued vectors (x1, x2, ..., xr) such that x1 +
x2+ ... + xr = n, which is equal to C(n + r  1, r  1).
Proof:
Immediate from Proposition 1.2.
(A note: terms equivalent to theorem include: fact, proposition, lemma, theorem,
corollary, …)
1- 9