Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Probability 1 Sample Space and Events Definition 1 A sample space, denoted S, is the collection of all outcomes when an experiment is performed. ¦ Example 1 Toss a coin two times, the sample space is S = {HH, HT, T H, T T } 4 Example 2 Toss a coin until a head appears, the sample space is S = {H, T H, T T H, T T T H, . . .} 4 Definition 2 An event is a set outcomes form the sample space. The events are usually denoted A, B, C, . . .. If we regard S as a set, then an event is a subset of S. ¦ Union and intersection of events - The union of two events A and B, denoted A ∪ B, is the set of all outcomes that are either in A or in B - The intersection of two events A and B, denoted A ∩ B, is the set of all outcomes that are both in A and in B - The complement of an event A, denoted A, is the set of all outcomes that are not in A Most of elementary operations on sets can be expressed in terms of union, intersection and complement. Example 3 A \ B is the set of outcomes that are in A but not in B; it is left as an exercise to show that A \ B = A ∩ B 4 Mutually exclusive and exhaustive events Definition 3 A set of events A1 , A2 , A3 , . . . is mutually exclusive if and only if Ai ∩ Aj = ∅ , i 6= j (1) ¦ Definition 4 A set of events A1 , A2 , A3 , . . . is exhaustive if A1 ∪ A2 ∪ A3 . . . = S (2) ¦ Definition 5 A set of events A1 , A2 , A3 , . . . is mutually exclusive and exhaustive (called also partition) if (1) and (2) are both satisfied. ¦ 1 2 Probability Basics Axiom 1 A probability, denoted P is an application, P : S −→ [0, 1], satisfying: a. P (φ) = 0, P (S) = 1, and b. If A1 , A2 , A3 , . . . is a set of mutually exclusive events, then P P (A1 ∪ A2 ∪ A3 . . .) = P (Ai ) = P (A1 ) + P (A2 ) + P (A3 ) + . . . ¢ Property 1 (basic rules) a. (union rule) P (A ∪ B) = P (A) + P (B) − P (A ∩ B) b. (complement rule) P (A) = 1 − P (A) 2 The proof of the above property is left as an exercise. Note that other basic rules of probability can be expressed in terms of the the union and the complement rules. Example 4 P (A \ B) = P (A) − P (A ∩ B) Proof: A = (A \ B) ∪ (A ∩ B) 4 (mutually exclusive events) P (A) = P (A \ B) + P (A ∩ B), hence P (A \ B) = P (A) − P (A ∩ B). Exercise 1 Let A, B and C be three events, show that P (A∪B ∪C) = P (A)+P (B)+P (C)−P (A∩B)−P (A∩C)−P (B ∩C)+P (A∩B ∩C) 5 Inclusion-Exclusion formula: More generally, if A1 , A2 , . . . , An are n events, then n n n X X X P (A1 ∪ A2 ∪ . . . ∪ An ) = P (Ai ) − P (Ai ∩ Aj ) + P (Ai ∩ Aj ∩ Ak ) . . . + i=1 n−1 (−1) i,j=1 i,j,k=1 P (A1 ∩ A2 ∩ . . . ∩ An ) Proposition 1 (not required for STAT230) Let[ An , n = 1, 2.3, . . . be a increasing set of events (A1 ⊆ A2 ⊆ A3 . . .), then P ( An ) = lim P (An ). n n→+∞ 2 Proof: Let B1 = A1 , B2 = A2 ∩ A1 , B3 = AS Bn = An ∩ An−1 . The events 3 ∩ A2 , . . ., S +∞ Bn , n = 1, 2.3, . . . are mutually exclusive, and n=1 Bn = +∞ n=1 An . By axiom 1,b), S Pk P+∞ S+∞ k P ( n=1 An ) = n=1 P (Bn ) = lim n=1 P (Bn ) = lim P ( n=1 Bn ) = lim P (Ak ). k→+∞ k→+∞ 2 k→+∞ 3 Counting Definition 6 A selection of k objects out of n objects without replacement and withn! out order is called combination, denoted Cnk , and defined by: Cnk = k!(n−k)! ¦ Definition 7 A selection of k objects out of n objects without replacement and with n! order is called permutation, denoted k Pn , and defined by: k Pn = (n−k)! ¦ Example 5 a) In how many different ways can a group of 2 persons be formed if 5 persons are available ? b) In how many different ways can a this be done if the group must contain a president and a vice-president ? 4 Property 2 The number of permutations of n distinct objects is n!. 2 Property 3 (Basic counting rule) If an experiment has n1 outcomes, and if another experiment has n2 outcomes, then the total number of outcomes is n1 × n2 . 2 Exercise 2 In how many different ways can a team of 2 men and 3 women be formed if 5 men and 7 women are available ? In how many ways can this be done if one man and one woman refuse to be together in the team ? 5 Exercise 3 In how many different ways can a group of 10 persons be seated in a row if 3 of them insist to be next to each other ? 5 Exercise 4 Find the probability of a full-house in a poker hand. 5 Permutations of indistinguishable objects Property 4 The number of permutations of n objects, from which n1 are the same, n2 are the same, ..., and nk are the same (n1 + n2 + . . . + nk = n) is n! n1 ! × n2 ! × . . . × nk ! 2 n2 Proof: There are Cnn1 choices for the first n1 objects, and Cn−n choices for the 1 nk n2 objects, ..., and Cn−n1 −n2 −...−nk−1 choices for the nk objects. By the basic counting rule, the total number of choices is n! nk n2 Cnn1 × Cn−n × . . . × Cn−n = 1 1 −n2 −...−nk−1 n1 ! × n2 ! × . . . × nk ! Example 6 How many different words can be formed by using the letters of the word MISSISSIPPI ? 4 Exercise 5 In how many different ways can 15 students be seated in 3 classes of size 5 each ? In how many ways can this be done if two of the students, Asma and Maya, want to be in the same class room ? 5 3 4 Conditional Probability Definition 8 Let A and B be two events with P (B) > 0. The conditional probability of A given B, denoted by P (A|B), is defined by P (A|B) = P (A ∩ B) P (B) ¦ Example 7 5 cards are drawn at random and without replacement from a deck of 52 playing cards. Find the probability that all 5 cards are spades if at least 3 cards are spades. 4 Solution: spades} P (A|B) = Let A={event all 5 cards are spades}, and B={at least 3 cards are 5 P (A ∩ B) P (A) C13 = = 3 2 4 1 5 P (B) P (B) C13 ∗ C39 + C13 ∗ C39 + C52 Proposition 2 The conditional probability P (·|B) is itself a probability. 2 Proof: a. P (φ|B) = P (φ ∩ B) P (φ) P (S ∩ B) P (B) = = 0 ; P (S|B) = = =1 P (B) P (B) P (B) P (B) b. Let A1 , A2 , A3 , . . . be mutually exclusive events, P ((A1 ∪ A2 ∪ A3 . . .) ∩ B) P (A1 ∪ A2 ∪ A3 . . . |B) = P (B) P ((A1 ∩ B) ∪ (A2 ∩ B) ∪ (A3 ∩ B) . . .) = P (B) P (A1 ∩ B) P (A2 ∩ B) P (A3 ∩ B) = + + + ... P (B) P (B) P (B) = P (A1 |B) + P (A2 |B) + P (A3 |B) + . . . The previous proposition says that all rules of probability still apply to conditional probability; for example, P (A|C) = 1 − P (A|C) Property 5 (Multiplication rule) a. P (A ∩ B) = P (A|B) × P (B) = P (B|A) × P (A), and more generally b. P (A1 ∩. . .∩An ) = P (A1 )×P (A2 |A1 )×P (A3 |A1 ∩A2 ) . . .×P (An |A1 ∩A2 ∩. . .∩An−1 ) 2 Example 8 Cards are drawn at random and without replacement from a deck of 52 cards. Find the probability that the sixth spade is drawn at the tenth draw. 4 4 Solution: spade} Let A={event five spades in the first nine cards}, and B={sixth card is P (A ∩ B) = P (A) ∗ P (B|A) = 5 5 4 C13 ∗ C39 8 ∗ 9 C52 43 Independence Definition 9 Two events A and B are independent if and only if P (A|B) = P (A) or P (B|A) = P (B). ¦ Proposition 3 Two events A and B are independent if and only if P (A ∩ B) = P (A) × P (B) 2 Property 6 If A and B are independent, then so are a) A and B b) A and B c) A and B 2 Proof: a) P (B|A) = 1 − P (B|A) = 1 − P (B) = P (B) parts b) and c) are left as an exercise. Proposition 4 If n events A1 , A2 , . . . , An are mutually independent, then P (A1 ∩ A2 ∩ . . . ∩ An ) = P (A1 ) × P (A2 ) × . . . × P (An ) 2 Note that the events A1 , A2 , . . . , An can be ind two by two but not mutually independent. Example 9 A fair coin is tossed twice. Let A1 ={first toss is head}, A2 ={second toss is head}, A3 ={the two tosses show up the same face} P (A1 ) = P (A2 ) = P (A3 ) = 12 , and P (A1 ∩ A2 ∩ A3 ) = P (A1 ∩ A2 ) = not mutually independent. 1 4 6= 1 8 = P (A1 )P (A2 )P (A3 ), hence A1 , A2 , A3 are But P (A1 ∩ A2 ) = 14 = P (A1 )P (A2 ), P (A2 ∩ A3 ) = P (A2 ∩ A1 ) = 14 = P (A2 )P (A3 ). P (A1 ∩ A3 ) = P (A1 ∩ A2 ) = 14 = P (A1 )P (A3 ) hence A1 , A2 , A − 3 are independent two by two. 6 4 Bayes Rule Proposition 5 (Total probability formula) Let A1 , A2 , . . . , An be a set of mutually exclusive and exhaustive events, with P (Ai ) > 0 for i = 1, 2, . . . , n. Then for any event B, n X P (B) = P (Ai )P (B|Ai ) i=1 2 5 Proof: P (B) = P (B ∩ S) = P (B ∩ (A1 ∪ A2 ∪ . . . ∪ An )) = P ((B ∩ A1 ) ∪ (B ∩ A2 ) ∪ . . . ∪ (B ∩ An )) = P (B ∩ A1 ) + P (B ∩ A2 ) + . . . + P (B ∩ An ) = P (B|A1 )P (A1 ) + P (B|A2 )P (A2 ) + . . . + P (B|An )P (An ) A1 A2 B A3 ... A4 Proposition 6 (Bayes rule) Let A1 , A2 , . . . , An be a set of mutually exclusive and exhaustive events, with P (Ai ) > 0 for i = 1, 2, . . . , n. Let B be any event with P (B) > 0, then P (B|Ai )P (Ai ) P (Ai |B) = n X P (Ai )P (B|Ai ) i=1 2 Proof: P (Ai |B) = P (Ai ∩ B) P (B|Ai )P (Ai ) P (B|Ai )P (Ai ) = = n X P (B) P (B) P (B|Ai )P (Ai ) i=1 Example 10 (Urn model) Three balls are selected at random and without replacement from an urn U1 containing 5 white balls and 4 blue balls, and put in an empty urn U2 . A ball is then drawn from U2 , its blue. Find the probability that all three balls transferred from U1 to U2 were blue. 4 Solution: Let Bi ={event i blue balls selected from U1 } (i = 0, 1, 2, 3), and B={event blue ball selected from U2 } P (B|B3 )P (B3 ) P (B|B0 )P (B0 ) + P (B|B1 )P (B1 ) + P (B|B2 )P (B2 ) + P (B|B3 )P (B3 ) 1 ∗ C43 3 = = 1 2 3 1 2 2 1 3 28 0 ∗ C5 + 3 ∗ C4 ∗ C5 + 3 ∗ C4 ∗ C5 + 1 ∗ C4 P (B3 |B) = 6