Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Chapter 3 Conditional Probability and Independence Wen-Guey Tzeng Computer Science Department National Chiao Tung University Conditional probability • P(A|B) = the probability of event A given the occurrence of event B • Knowing that B has occurred changes the probability of A’s occurrence A S • Definition. If P(B)>0, the conditional probability of A given B, P( AB ) P( A | B ) P( B ) 2014 Fall 2 • • • • A class of 80 students. 50 of them take Linear Algebra, 60 of them take Probability, and 30 of them take both. Randomly choose a student and find that the student takes Probability What is the probability that the student takes Linear Algebra also? Sol: Let B be event that a selected student takes Probability. Let A be the event that a selected student takes Linear Algebra. P(A|B )= P(AB)/P(B) = (30/80) / (60/80) 2014 Fall 3 • • Draw 8 cards from a deck of 52 cards. Given 3 of them are spades, what is P(the remaining 5 are also spades|3 are spades)=? Sol: B: the event that at least 3 of them are spades A: the event that the remaining 5 are spades P(A|B) = P(AB)/P(B) = 5.44 x 10-6 P(AB)=C(13,8)/C(52,8) P(B)=[C(13,3)C(39,5)+C(13,4)C(39,4)+ …+C(13,8)] / C(52,8) 2014 Fall 4 • Box 1: 2 red balls and 3 green balls • Box 2: 4 red balls and 1 green balls • Choose a box with P(Box 1)=0.6, P(Box 2)=0.4, and then randomly choose two balls from the box. • What is the sample space? • What is the probability of • the event that Box 1 is chosen and two balls are both red? • the event that two red balls are chosen under the condition that Box 2 is chosen? • the event that one red and one green are chosen? • the event that Box 1 is chosen under the condition that two balls are red and green? 2014 Fall 5 P(BiDj) D1: balls are GG D2: balls are RG D3: balls are GG B1: Box 1 is chosen B2: Box 2 is chosen 2014 Fall 6 • • • From the set of all families with 2 children, a family is selected at random and is found to have a girl. Assume that in a 2-child family all sex distributions are equally probable. What is the probability that the other child of the family is a girl? Sol: All possible outcomes (equally likely): bb, bg, gb, gg Let B be the event that the family has a girl. Let A be the event that the other child of the family is a girl. P(A|B)=P(AB)/P(B)=P({gg} / P({bg,gb,gg})=(1/4)/(3/4)=1/3. 2014 Fall 7 • • • From the set of all families with 2 children, a child is randomly selected and found to be a girl. Assume that in a 2-child family all sex distributions are equally probable. What is the probability that the other child of the family is a girl? Sol: Let B be the event that a randomly selected child is a girl. Let A be the event that the second child of the family is a girl. P(A|B)= P(AB)/P(B) = (1/4)/(2/4)=1/2. How do you define a sample space for this problem? 2014 Fall 8 • Conditional probability = reduction of sample space S A C S’ = B 2014 Fall 9 All probability theorems hold under conditional probability, for P(B)0, • P(Ac|B) = 1-P(A|B) • P(EF|B) = P(E|B)+P(F|B)-P(EF|B) • IF C A, P(C|B) P(A|B) • Inclusion-exclusion principle •… 2014 Fall 10 • A child mixes 10 good and 3 dead batteries. • To find the dead battery, the father tests batteries one by one and without replacement. • If the first 4 batteries are good, what is the probability that the fifth is dead? Sol: Old sample space S = {all sequences of 10 good and 3 dead batteries} New sample space S’ = {all sequences of 6 good and 3 dead batteries} P(the fifth is dead | the first 4 are good} -- in the old sample space S = P(the first is dead) -- in the new sample space S = 3/9 2014 Fall 11 • There are three boxes. One contains a 100-dollar bill and the other two are empty. • You select one of them randomly. • What is the probability that you win 100 dollars? ---------------------• I open an empty box that is not selected by you. • Will you switch to the other un-opened box ? • Yes, always switch. Why? • No, never switch. Why? ----------------------- 2014 Fall 12 Sol: • Assume that the bill is in box 1. – reduce the sample space • Let outcome=(x, y, z) • x: your first choice • y: the opened box • z: the final choice (switched to) • Yes: always switch • The reduced sample space S={ (1, 2, 3), (1, 3, 2), (3, 2, 1), (2, 3, 1)} • P({3,2,1}) = P({2,3,1}) = P({1,2,3), (1,3,2)}) = 1/3 • P(you win)=P({(3,2,1), (2,3,1)})=2/3 • No: never switch • The reduced sample space S={(1,2,1), (1,3,1), (2,3,2), (3,2,3)} • P(you win) = P({(1,2,1), (1,3,1)} = 1/3 2014 Fall 13 Law of multiplication P( AB ) P( A | B ) P( B ) P( AB) P( B) P( A | B) P(AB) P(BA) P(B)P(A|B) P(A)P(B|A) Extension P(A1 A2 A3 ) P(A1 )P(A2|A1 )P( A3 | A1 A2) 2014 Fall 14 • Suppose 5 good fuses and 2 defective ones have been mixed up. To find the defective fuses, we test them 1-by-1, at random and without replacement. • What is the probability that we are lucky and find both of the defective fuses in the first two tests? Sol: Let D1 and D2 be the events of finding a defective fuse in the 1st and 2nd tests, respectively. P(D1D2) = P(D1)P(D2|D1) = 2/7x1/6 = 1/21 P(find both defective fuses in exactly 3 tests) = ? 2014 Fall 15 Law of total probability Theorem (Law of total probability) Let P(B)>0, and P(Bc)>0, then P(A) = P(A|B)P(B) + P(A|Bc)P(Bc) Proof: P(A) = P(AB) + P(ABc) and use the law of multiplication. 2014 Fall 16 • An insurance company rents 35% of the cars for its customers from agency I and 65% from agency II. If 8% of the cars of agency I and 5% of the cars of agency II break down during the rental periods, what is the probability that a car rented by this insurance company breaks down? Sol: P(A) = P(A|I)P(I)+ P(A|II)P(II) = (0.08)(0.35)+(0.05)(0.65) = 0.0605 Table method? Tree diagram method? 2014 Fall 17 • Gambler’s ruin problem 2 gamblers play the game of “heads or tails,” in which each time a fair coin lands heads up, player A wins $1 from B, and each time it lands tails up, player B wins $1 from A. Suppose that the player A initially has ‘a’ dollars and player B has ‘b’ dollars. If they continue to play this game successively, what is the probability that • A will be ruined ? • the game goes forever with nobody winning ? 2014 Fall 18 Sol: • Sample space ? • Let E be the event that A will be ruined if he starts with i dollars, and let pi=P(E). Our aim is to calculate pa. To do so we define F to be the event that A wins the 1st game. Then P(E) = P(E|F)P(F) + P(E|Fc)P(Fc) => pi = pi+1(1/2)+pi-1(1/2) => pi+1 – pi = pi – pi-1 2014 Fall 19 p0=1, pa+b=0, and let p1- p0 = x pi+1 – pi = pi – pi-1 = … = p2- p1 = p1- p0 = x Thus p1 = p0 + x p2 = p0 + 2x … pi = p0 + ix … and pa+b = p0 + (a+b)x => x = -1/(a+b) So pi = 1 – i/(a+b). In particular pa = b/(a+b) 2014 Fall 20 • The same method can be used with obvious modifications to calculate qi, the probability that B is ruined if he starts with i dollars. qi = 1 – i/(a+b) Since B starts with b dollars, he will be ruined with probability qb = a/(a+b). • Thus the probability that the game goes on forever with nobody winning is 1-(qb+pa)=0. 2014 Fall 21 • Extension If {B1, B2, … , Bn} is a partition of the sample space S of an experiment and P(Bi)>0 for all i. Then, for any event A of S, P(A) = P(A|B1)P(B1) + P(A|B2)P(B2) + … + P(A|Bn)P(Bn) 2014 Fall 22 • Suppose that 80% of the seniors, 70% of the juniors, 50% of the sophomores, and 30% of the freshmen of a college use the library of their campus frequently. If 30% of all students are freshmen, 25% are sophomores, 25% are juniors, and 20% are seniors, what percent of all students use the library frequently? Sol: P(A)= P(A|F)P(F) + P(A|O)P(O) + P(A|J)P(J) + P(A|E)P(E) = (0.30)(0.30) + (0.50)(0.25) + (0.70)(0.25) + (0.80)(0.20) = 0.55 2014 Fall 23 Bayes’ formula •𝑃 𝐵𝐴 = 𝑃(𝐴|𝐵)P B P(𝐴) = P(A|B)P B P(A|B)P B +P(A|𝐵𝑐 )P(𝐵c ) • A conditional probability formula with very important applications on Statistics. • Prior probability: P(B), P(Bc) – the probability of event (hypothesis) B is believed originally • Likelihoods (observation probability) : P(A|B), P(A|Bc) • Posterior probability: P(B|A) – the probability of B under that A is observed 2014 Fall 24 • In a bolt factory, 30% and 70% of production is manufactured by machines I and II, respectively. • 4% and 6% of the output of machines I and II are defective. • What is the probability that a randomly selected bolt that is found to be defective is manufactured by machine I? probability A: defective Ac: non-defective B: manufactured by I Bc: manufactured by II We want to compute P(B|A)? Prior probability P(B)=0.3 Likelihoods: P(A|B)=0.04, P(A|Bc)=0.06 Post probability: P(B|A)=0.04*0.3/(0.04*0.3+0.06*0.7)=0.222 2014 Fall 25 Tree diagram method ? 2014 Fall 26 • • • • In a double homicide, a suspect, John, is caught. The jury believes that John is guilty with 15%。 Later, a DNA sample is found and match John’s DNA By forensic estimation, the probability that the DNA does not come from John is 10-9. • How certain should the jury be about that John is guilty? Sol: • G: the event that John is guilty, P(G) = 0.15 I: the event that John is innocent, P(I)=0.85 • D: the event that the DNA matches John’s DNA P(D|G)=1, P(D|I)=10-9 • Compute P(G|D)= 2014 Fall 27 • A car insurance company needs to decide whether to accept a car insurance from a driver. • By estimation, the percentage of good drivers is 80% • The probability that a good driver has a car accident within a year is 10%, and the probability that a bad driver has a car accident within a year is 40% 2014 Fall 28 • What is the probability that a driver who has a car accident in the first year is a good driver? 2014 Fall 29 • What is the probability that a driver who has a car accident in the first year and no car accident in the second year is a good driver? 2014 Fall 30 • What is the probability that a driver who has no car accidents in two consecutive years is a bad driver? 2014 Fall 31 Extension: B1, B2, …, Bn partition S. P A 𝐵𝑘 P 𝐵𝑘 𝑃(𝐵𝑘 |𝐴) = P 𝐴 𝐵1 P 𝐵1 + ⋯ + P A 𝐵𝑛 P(𝐵𝑛 ) • Prior probability: P(Bk), 1kn • Likelihoods: P(A|Bi), 1in • Post probability: P(Bk|A ) 2014 Fall 32 • A box contains 7 red and 13 blue balls. • 2 balls are selected at random and are discarded without their colors being seen. • A 3rd ball is drawn randomly and observed to be red, • What is the probability that both of the discarded balls were blue? Sol: • B1: discarded balls are RR, P(B1) = B2: discarded balls are RB, P(B2) = B3: discarded balls are BB, P(B3) = • A: the third ball is R • P(A|B1)= P(A|B2)= P(A|B3)= • Want to compute P(B3|A) 2014 Fall 33 Independence Two events A and B are independent if P(AB)=P(A)P(B) A B A B A B 2014 Fall 34 • Theorem:A and B are independent if either one of the following holds • P(AB) = P(A)P(B) • P(A|B) = P(A) • P(B|A) = P(B) • Theorem: If A and B are independent, so are • A and Bc • Ac and B • Ac and Bc • What is good for independence? • easy to compute 2014 Fall 35 • • • • A card is drawn from a deck of 52 cards. A: the event that an ace is drawn. B: the event that a heart is drawn. Are A and B independent? • P(A) = • P(B)= • P(AB) = 2014 Fall 36 • An urn contains 5 red and 7 blue balls. Suppose that 2 balls are selected at random with/without replacement. Let A and B be the events that the first and the second ball are red, respectively. Are A and B independent? • With replacement • • • • P(A) = P(B) = P(AB) = P(B|A)P(A) = Independent ? • Without replacement • P(B|A) = • P(B) = P(B|A)P(A)+P(B|Ac)P(Ac) = • Dependent ? 2014 Fall 37 Jailer’s paradox • Three prisoners Alex, Bill, and Tim. One of them is condemned to death. The other two will be freed. • The jailor and the judge know who is condemned to death. • Alex has written a letter to his fiancée and wants to give it to either Bill or Tim, whoever goes free, to deliver. • Alex asks the jailer to tell him which of the two will be freed. • The jailer refuses to give that information to Alex, explaining that, if he does, the probability of Alex dying increases from 1/3 to 1/2. Really? 2014 Fall 38 Sol-1: • Let A, B, and T be the events that “Alex dies,” ”Bill dies,” and “Tim dies.” • Let w1=(T, the jailer tells Alex that Bill goes free) w2=(B, the jailer tells Alex that Tim goes free) w3=(A, the jailer tells Alex that Bill goes free) w4=(A, the jailer tells Alex that Tim goes free) • The sample space S = {w1, w2, w3, w4}. P(w1)=P(w2)=1/3, P(w3)=P(w4)=1/6 • Let J be the event that “the jailer tells Alex that Tim goes free” • Then, 1 P( A | J ) 2014 Fall P ( AJ ) P ( w4 ) 1 6 1 1 P( J ) P ( w2 ) P ( w4 ) 3 3 6 39 Sol-2: • Zweifel, in June 1986 issue of Mathematics Magazine, page 156 • He analyzes this paradox by using Bayes’ formula: P( A | J ) P ( J | A) P ( A) P ( J | A) P ( A) P ( J | B ) P ( B ) P ( J | C ) P (C ) 1 1 2 3 1 1 1 1 1 0 2 3 3 3 1 3 2014 Fall 40 Remark: • If A and B are disjoint (mutually exclusive), then A and B are dependent. • If A B, then A and B are dependent 2014 Fall 41 Extension • Events A, B, and C are independent if the following all hold: • • • • P(AB) = P(A)P(B) P(AC) = P(A)P(C) P(BC) = P(B)P(C) P(ABC) = P(A)P(B)P(C) 2014 Fall 42 • We draw cards, one at a time, at random and successively from an ordinary deck of 52 cards with replacement. What is the probability that an ace appears before a face card? Sol: • Let E be the event of an ace appearing before a face card. • Let A, F, and B be the events of ace, face card, and neither in the first experiment, respectively. Then • P(E) = P(E|A)P(A) + P(E|F)P(F) + P(E|B)P(B) = 1(4/52) + 0(12/52)+ P(E|B)(36/52) P(E|B) = P(E) P(E)=4/52+P(E)(36/52) • So, P(E)=1/4 2014 Fall 43