Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
An Introduction to Probability (a monograph) 機率學 蔡文祥 講座教授 Wen-Hsiang Tsai, Ph. D. Chair Professor, NCTU Department of Computer Science National Chiao Tung University February 2012 Chapter 1 Combinatorial Analysis 1.0 Introduction to Probability Theory Why study the probability theory? -- Probability is a common sense for scholars and people in modern days. No engineer or scientist can conduct research and development works without knowing the probability theory. Academic fields based on the probability theory --A lot of academic fields are based on the probability theory: statistics communication theory computer performance evaluation signal and image processing pattern recognition queuing theory stochastic processes quality control game theory .... Applications of the probability theory --A lot of applications are based on the probability theory: character recognition speech recognition opinion survey missile control weather prediction seismic analysis ... Course requirement and others -- Prerequisite course: calculus. Grade evaluation: home works 25%; two midterm exams + one final, each 25%. Office hours: right before and after class at the classroom or Computer Vision Lab (EC129). 1- 1 Reference textbook --Sheldon Ross, A First Course in Probability, 8th Edition, Prentice-Hall International Inc., NJ, U. S. A., 2009. Some interesting questions --Do you have answers for the following questions? Is there a strategy to win in playing a slot machine in a casino? Do you know the meanings of people’s opinion polls reported in daily newspapers? With the election fast approaching, the Taiwan Thinktank has released its latest opinion poll figures. KMT presidential candidate Ma Ying-jeou has a support rate of 39.5 percent, with DPP candidate Tsai Ing-wen a close second, at 39.1 percent, while PFP candidate James Soong lags at 11.1 percent. The polls were conducted on Dec. 23 and 24, canvassing opinion from 1,067 adults above the age of 20. The confidence level of the poll is 95 percent and margin of error 3 percent. (2011/12/26) (大選腳步逼近,台灣智庫選情分析,公佈剛出爐的民調,馬英九拿到百分之 39.5 的支持度, 蔡英文也有百分之 39.1,宋楚瑜得到百分之 11.1,這份在 12/23、24 日進行的民調,電話抽查全 國 20 歲以上民眾,完成 1067 份樣本,信心水準 95%抽樣誤差 3%。) Are there really pre-known winning numbers (明牌) for lotto games? 1- 2 1.1 Introduction to Combinatorial Analysis Combinatorial analysis is a mathematical theory of counting. Many problems in probability can be solved by simply counting the number of different ways in which a certain event can occur. 1.2 Principle of Counting Some terms -- experiment: any human activity, like tossing a die (骰子). outcome: a result of an experiment, like 5 points on a side of a die. Basic product principle of counting --“If experiment 1 has m possible outcomes and if for each outcome of experiment 1, experiment 2 has n possible outcomes, then an experiment of performing experiment 1 and experiment 2 simultaneously, called experiment 1 & 2, has mn possible outcomes.” Proof: Easy; omitted. Example 1.1 --In tossing simultaneously a coin with two sides A and B and a die with six sides 1 through 6, how many possible outcomes will appear? Solution: The tossing may be regarded as an experiment (experiment 1 & 2) consisting of two smaller experiments, tossing a coin (experiment 1) followed by tossing a die (experiment 2). Experiment 1 has 2 outcomes. Experiment 2 has 6 outcomes. So experiment 1 & 2 has 26 = 12 outcomes. Generalized product principle of counting --“If Experiments 1 through k have n1 through nk outcomes, respectively, then the experiment 1 & 2 & … & k has n1n2…nk outcomes.” Proof: Easy to derive from the basic principle by induction. Basic sum principle of counting --“If Experiment 1 has m possible outcomes and if experiment 2 has n possible outcomes, then an experiment which might be experiment 1 or experiment 2, called experiment 1 or 2, has m + n possible outcomes.” Proof: Easy; omitted. Example 1.2 --In tossing an object which might be a coin (with two sides A and B) or a die (with six sides 1 through 6), how many possible outcomes will appear? Solution: 1- 3 The tossing may be tossing a coin (experiment 1) or tossing a die (experiment 2), or just experiment 1 or 2. So the number of outcomes is 2 + 6 = 8 according to the above basic sum principle of counting. Generalized sum principle of counting --“If Experiments 1 through k have respectively n1 through nk outcomes, then the experiment 1 or 2 or … or k has n1 + n2 + … + nk outcomes. Proof: Easy to derive from the basic principle by induction. 1.3 Permutations Fact 1.1: permutations of n objects --The number of different permutations of n different objects is n! n(n 1)(n 2)…21. where “” means “equal to by definition.” (Note: “n!” is pronounced as “n factorial.”). Proof: The operation of finding all permutations may be regarded as an integrated experiment of filling n objects into n slots as follows. Experiment 1: filling an object into the 1st slot with n choices (outcomes). Experiment 2: filling an object into the 2nd slot with n 1 choices (outcomes). … Experiment n: filling an object into the nth slot with 1 choice (outcome). So, according to the generalized product principle of counting, this integrated experiment has the following number of outcomes (i.e., the following number of permutations): n(n 1)(n 2)…21 = n!. Fact 1.2: permutations of r objects selected from n ones --The number of different permutations of r objects selected from n different objects is P(n, r) n! . (n r )! Proof: The operation of finding all permutations of r objects selected from n different objects may be regarded as an integrated experiment of filling r objects into n slots as follows. Experiment 1: filling an object into the 1st slot with n choices (outcomes). 1- 4 2: filling an object into the 2nd slot with n 1 choices (outcomes). Experiment … r: filling an object into the rth slot with n (r 1) choices (outcomes). So, the use of the generalized product principle of counting leads to the following number of permutations: Experiment n(n 1)(n 2)…[n (r 1)] = [n(n 1)(n 2)…21] / [(n r)(n r 1)…21] = (n!)/(n r)! Done. Example 1.3 --In permuting the six letters in the word “avatar” (a title used for the movie “阿 凡達”), how many possible outcomes will appear? Solution: Let the three a’s in the word be distinguished as a1, a2 and a3, respectively. Then all the six letters are different, so the number of permutations of them (called labeled permutations) is n! = 6!. However, consider each of the real permutations without distinguishing the three a’s --- e.g., w = ratava. (Note: the Latin word e.g. means “for example.”) The following are all of the 6 (=3!) labeled permutations among the 6! ones, which come from permuting the three labeled a’s in w = ratava: ra1ta2va3, ra1ta3va2, ra2ta1va3, ra2ta3va1, ra3ta1va2, ra3ta2va1. All these six labeled permutations should be considered as an identical real permutation, which is w = ratava. Since each real permutation has six of such labeled permutations coming from the three a’s, we conclude that the desired number of real permutations is just 6!/3! = 654 = 120. Fact 1.3: permutations of indistinguishable objects --The number of different permutations of n objects with n1 alike, n2 alike, ..., nr alike and n = n1 + n2 + ... + nr is n! . n1 !n2 !... nr ! (Note: alike means the objects in a group are indistinguishable from one another.) Proof: By the same reasoning used in Example 1.3, we know that the number of labeled permutations is n!. 1- 5 Also, according to the generalized product principle of counting, each real permutation has n1!n2!...nr! corresponding labeled permutations because there are n1 objects alike, n2 objects alike, …, and nr objects alike. Therefore, the number of real permutations is (n!)/(n1!n2!...nr!). 1.4 Combinations Fact 1.4: basic formula for combinations --The number of different groups of r items that could be formed from a set of n distinct objects with the order of selections being ignored is n n! C(n, r) . r (n r )!r ! (Note: the objects selected to be in a group are regarded as indistinguishable.) Proof: Since in the formation of each desired r-item group, the n objects are divided into two groups (one with r items and the other with n r items), with all the objects in either group being regarded as indistinguishable. So, the problem may be considered as one of finding permutations from n objects with r ones alike and the remaining (n r) ones also alike (because objects in a group are considered as indistinguishable). Therefore, according to Fact 1.3 above, the desired number is just n! . (n r )!r ! Example 1.4 --From a group of 5 men and 7 women, how many different committees consisting of 2 men and 3 women can be formed? Solution: Experiment 1: select 2 men from 5. Experiment 2: select 3 women from 7. Experiment of forming a committee: experiment 1 & 2. #possible outcomes of experiment 1 = C(5, 2) = (54)/(21) = 10. (Note: we use # to mean “number of.”) #possible outcomes of experiment 2 = C(7, 3) = (765)/(321) = 35. #possible outcomes of experiment 1 & 2 = C(5, 2)C(7, 3) = 350 by the basic product principle of counting That is, the desired number of possible outcomes of the experiment of forming a committee is 350. Fact 1.5: two identities about computations of combinations --The following two identities are true: 1- 6 C(n, r) = C(n, n r); C(n, r) = C(n 1, r 1) + C(n 1, r). Proof: as exercises. Binomial Theorem --The following identity is true: (x + y)n = n C (n, k )x k y nk . k 0 Proof: The proof can be done by combinatorial analysis. Consider (x + y)n as a product of labeled x’s and y’s: (x + y)n = (x1 + y1)(x2 + y2)…(xn + yn). For example, we have (x1 + y1)(x2 + y2)(x3 + y3) = (x1x2 + x1y2 + y1x2 + y1y2)(x3 + y3) = x1x2x3 + x1y2x3 + y1x2x3 + y1y2x3 + x1x2y3 + x1y2y3 + y1x2y3 + y1y2y3 = x3 + x2y + x2y + x2y + … + yyy (with no label) There are in general 2n terms (like x1x2x3) in the result (for the above example, there are 23 = 8 terms). In each term (like x1x2y3), there are n factors (like x1, x2, y3 in x1x2y3) and there is one factor for each i = 1, 2, …, n (like x2 for i = 2). Among the 2n terms, how many of them will contain k of the x’s and n k of the y’s (k 個 x 以及 n k 個 y)? Since there are n factors in each term for selection of k x’s, the answer is C(n, k) according to Fact 1.4 (basic formula for combinations). That is, we have C(n, k) terms of the form xkynk, or equivalently, we have the item C(n, k)xky n k in the result, where k may take the values of 0 through n. Therefore, by the basic sum principle of counting, the final result is (x + y)n = n C (n, k )x k y nk . k 0 1.5 Multinomial Coefficients Fact 1.6: divisions of n objects into r groups --The number of different ways of dividing a set of n distinct objects into r distinct groups of respective sizes n1, n2, ..., nr is C(n; n1, n2, …, nr) Proof: 1- 7 n! . n1! n2 !... nr ! The total operation can be divided into r steps, one following another, meaning that the generalized product principle of counting is applicable. First, choose n1 objects from the n ones, and the number of ways to do this is N1 = n!/n1!(n n1)!. Then, choose n2 objects from the remaining n n1 ones, and the number of ways to do this is N2 = (n n1)!/n2!(n n1 n2)!. … Finally, choose nr objects from the remaining n n1 n2 … nr1 = nr ones and the number of ways to do this is Nr = (n n1 n2 … nr1)!/nr!(n n1 n2 … nr1 nr)! = (n n1 n2 … nr1)!/nr!0!. (Note: 0! = 1 by definition.) Then, by applying the generalized product principle of counting, we get the following desired number of divisions after some reductions: N1N2…Nr = [n!/n1!(n n1)!][(n n1)!/n2!(n n1 n2)!]… [(n n1 n2 … nr1)!/nr!(n n1 n2 … nr1 nr)!] n! = C(n; n1, n2, …, nr). n1 ! n2 !...nr ! ***Multinomial theorem --(*** means “may be skipped”) The following identity is true: (x1 + x2 + … + xr)n = C (n; n1 , n2 ,..., nr )x1n1 x2n2 ...xrnr ( n1 , n2 ,..., nr ): n1 n2 ... nr n where the notation (n1, n2, …, nr): n1+n2+…+nr = n under the summation symbol means that the sum is performed over all possible nonnegative integer-valued vectors (n1, n2, …, nr) such that n1 + n2 + … + nr = n. Proof: as an exercise. 1.6 On Distribution of Balls in Urns Fact 1.7: ways of distributing n distinguishable balls into r urns --The number of ways for n distinguishable balls to be all distributed into r urns is: r r ... r r n . n times Proof: Each ball has r urns to choose. 1- 8 The operation may be regarded as n experiments in a sequence. So, by the generalized product principle of counting, we get the desired number of ways to be r r ... r r n . n times Proposition 1.1 --The number of positive integer-valued vectors (x1, x2, ..., xr) such that x1 + x2 + ... + xr = n is C(n 1, r 1). Proof. The problem is equivalent to choosing r 1 spaces in n 1 ones as illustrated in the following diagram so that the #objects between every two neighboring chosen spaces (specified by the symbol ^) becomes nonzero: o^o o^o o ... o. n objects So, the number of ways for doing this is C(n 1, r 1). Proposition 1.2 --The number of nonnegative integer-valued vectors (x1, x2, ..., xr) such that x1 + x2 + ... + xr = n is C(n + r 1, r 1). Proof. To meet the requirement of “nonnegativeness,” let yi = xi + 1 so that yi becomes positive and y1 + y2 + ... + yr = n + r. Then, we can use Proposition 1.1 to derive the desired result, namely, C(n + r 1, r 1). Fact 1.8: ways of distributing n indistinguishable balls into r urns --The number of ways for n identical balls to be distributed into r urns is just the number of nonnegative integer-valued vectors (x1, x2, ..., xr) such that x1 + x2+ ... + xr = n, which is equal to C(n + r 1, r 1). Proof: Immediate from Proposition 1.2. (A note: terms equivalent to theorem include: fact, proposition, lemma, theorem, corollary, …) 1- 9