Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
1 PROBABILITY AND COMPUTING RANDOMIZED ALGORITHMS AND PROBABILISTIC ANALYSIS CHAPTER 1 IWAMA and ITO Lab. M1 Sakaidani Hikaru About the Book 2 Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 Chapter 8 Chapter 9 Chapter 10 Chapter 11 Corematerial Chapter 12 Chapter 13 Chapter 14 About the Book 3 Chapter 1 Chapter 2 Chapter 3 Additional Chapteradvanced 4 material Chapter 5 Chapter 6 Chapter 7 Chapter 8 Chapter 9 Chapter 10 Chapter 11 Chapter 12 Chapter 13 Chapter 14 About the Book 4 Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 Chapter 8 Review elementary Chapter 9 probability theory Chapter 10 Chapter 11 Chapter 12 Chapter 13 Chapter 14 About the Book 5 Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 Chapter 8 Chapter 9 Chapter 10 Chapter 11 Chapter 12 More advanced topics Chapter 13 Chapter 14 About the Book 6 Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 Chapter 8 Chapter 9 Chapter 10 Chapter 11 Chapter 12 Chapter 13 Chapter 14 CHAPTER 1 Events and probability 7 1.1 Verifying Polynomial Identities 1.2 Axioms of Probability 1.3 Verifying Matrix Multiplication 1.4 A Randomized Min-Cut Algorithm CHAPTER 1 Events and probability 8 1.1 Verifying Polynomial Identities 1.2 Axioms of Probability 1.3 Verifying Matrix Multiplication 1.4 A Randomized Min-Cut Algorithm Verifying Polynomial Identities 9 The problem of verifying the following identity ? More generally… ? : canonical form Verifying Polynomial Identities 10 First method (deterministic algorithm) ? It takes multiplications Verifying Polynomial Identities 11 There are at least two problems This approach is to write another program that does the same thing the first program to do. 1. If there is a bug in the first program, the same bug may occur in the checking program. 2. we would like to check the answer in less time than it takes to solve. Verifying Polynomial Identities 12 Second method (randomized algorithm) First, the algorithm chooses an integer uniformly at random in the range of {1, … , 100d} (d : degree of F(x)) If the algorithm decides If the algorithm decides But this algorithm may give a wrong answer. → How can the algorithm give the wrong answer? Verifying Polynomial Identities 13 If If , the algorithm gives the correct answer. , if , the algorithm gives the correct answer, if , the algorithm gives the wrong answer. For this, must be a root of has no more than . roots. is chosen in the range of {1, … , 100d} . → the chance that the algorithm returns a wrong answer is no more than 1/100. CHAPTER 1 Events and probability 14 1.1 Verifying Polynomial Identities 1.2 Axioms of Probability 1.3 Verifying Matrix Multiplication 1.4 A Randomized Min-Cut Algorithm Axioms of Probability 15 Definition 1.1 A probability space has three components: 1. a sample space : the set of all possible outcomes; 2. a family of sets representing the allowable events : each set in is a subset of ; 3. a probability function satisfying Definition 1.2. Axioms of Probability 16 Definition 1.2 A probability function is any function that satisfies the following conditions: 1. for any event ; 2. ; 3. for any finite or countably infinite sequence of pairwise mutually disjoint events Axioms of Probability 17 Because events are sets, : the occurrence of both and : the occurrence of either or : the occurrence of an event that is in but not in Axioms of Probability 18 Lemma 1.1 For any two events Proof and Axioms of Probability 19 Lemma 1.2 For any finite or countably infinite sequence of events Axioms of Probability 20 Lemma 1.3 : inclusion-exclusion principle Let be any events. Then Axioms of Probability 21 The randomized algorithm for verifying polynomial identities Let represent the event that the algorithm failed to give the correct answer. It may seem unusual to have an algorithm that can return the wrong answer. speed trade-off correctness Axioms of Probability 22 The randomized algorithm for verifying polynomial identities Let represent the event that the algorithm failed to give the correct answer. The algorithm give the correct answer 99% of the time when the polynomials are not equivalent. Can we improve this probability? Axioms of Probability 23 One way to improve the probability the algorithm chooses an integer uniformly at random in the range of {1, … , 1000d} → then the probability of a wrong answer is at most 1/1000 But the range of values we can use is limited by the precision available on the machine. Axioms of Probability 24 Another approach Repeat the algorithm multiple times The algorithm has one-sided error. The algorithm may be wrong only when it outputs If the algorithm finds in at least one round of the algorithm, we know . Only if there is equality for all runs, the algorithm outputs . Axioms of Probability 25 Repeatedly choosing random numbers according to a given distribution is generally referred to as sampling. In this case, we can choose random numbers in the range of {1, … ,100} in two ways: 1. with replacement 2. without replacement Axioms of Probability 26 The case where sampling is done with replacement We repeat the algorithm k times. The input polynomials are not equivalent. What is the probability that the algorithm give a wrong answer? Axioms of Probability 27 Definition 1.3 Two events and are independent if and only if More generally, events are mutually independent if and only if, for any subset , Axioms of Probability 28 Let be the event that, on the ith run of algorithm, we choose a root such that . The probability that the algorithm returns the wrong answer is given by Axioms of Probability 29 The case where sampling is done without replacement We repeat the algorithm k times. The input polynomials are not equivalent. What is the probability that the algorithm give a wrong answer? Axioms of Probability 30 Definition 1.4 The conditional probability that event that event occur is When and are independent and occurs given , Axioms of Probability 31 Let be the event that the random number chosen in the ith iteration of the algorithm is a root of . The probability that the algorithm returns the wrong answer is given by Axioms of Probability 32 In the th iteration of the algorithm, there are only values out of the remaining choices Axioms of Probability 33 The probability that the algorithm returns the wrong answer is given by CHAPTER 1 Events and probability 34 1.1 Verifying Polynomial Identities 1.2 Axioms of Probability 1.3 Verifying Matrix Multiplication 1.4 A Randomized Min-Cut Algorithm Verifying Matrix Multiplication 35 We want to verify whether ( are matrices) (we are working over the integers modulo 2.) One way is to multiply and and compare the result to . The simple matrix multiplication algorithm takes operations. More sophisticated algorithm takes operations. Verifying Matrix Multiplication 36 We use a randomized algorithm. The If algorithm chooses a random vector , then Otherwise , it returns that . The algorithm can be done in time obvious way. . in the Verifying Matrix Multiplication 37 Theorem 1.4 If and if is chosen uniformly at random from , then Verifying Matrix Multiplication 38 Lemma 1.5 Choosing uniformly at random is equivalent to choosing each independently and uniformly {0, 1}. Proof If each is chosen independently and uniformly at random, then each of the possible vector is chosen with probability , giving the lemma. Verifying Matrix Multiplication 39 Let . Since it must have some nonzero entry. Let that entry be . If For and , then . , it must be the case that Verifying Matrix Multiplication 40 Instead of reasoning about the vector , we choose the in order, from down to . Consider the situation just before is chosen. The equality holds with probability at most ½. The probability that is at most ½. This idea is called the principle of deferred decisions. Verifying Matrix Multiplication 41 Theorem 1.6 [Law of Total Probability] Let be mutually disjoint events in the sample space , and let . Then Verifying Matrix Multiplication 42 Proof of Theorem 1.4 Verifying Matrix Multiplication 43 Theorem 1.4 If and if is chosen uniformly at random from , then Verifying Matrix Multiplication 44 This algorithm has a one-sided error. Run the algorithm multiple times. The running time is The probability of error is at most Verifying Matrix Multiplication 45 Theorem 1.7 [Bay’s Law] Assume that are mutually disjoint sets such that . Then Verifying Matrix Multiplication 46 Problem We are given three coins. Two of the coins are fair. One of the coins is biased, landing heads with probability 2/3. Flip each the coins. The first and second coins come up heads, and the third comes up tails. What is the probability that the first coin is the biased one? A B C Pr(heads) 1/2 1/2 2/3 Pr(tails) 1/2 1/2 1/3 1st coin 2nd coin 3rd coin heads heads tails Verifying Matrix Multiplication 47 Let be the event that the ith coin flipped is the biased one. Let be the event that the three coin flips come up heads, heads, and tails. Before flips, Verifying Matrix Multiplication 48 We can compute the probability of the event conditioned on Applying Bay’s law, Verifying Matrix Multiplication 49 In the matrix multiplication case, if we have no information, a reasonable prior assumption is that the identity is correct with probability ½. Let be the event that the identity is correct. Let be the event that the test returns the identity is correct. Verifying Matrix Multiplication 50 If we run the randomized test ones and it returns that the matrix identity is correct, ∵ Verifying Matrix Multiplication 51 We run the randomized test again and it returns that the identity is correct Verifying Matrix Multiplication 52 In general : If our prior model is that and if the test returns that identity is correct, then Thus, if all 100 calls return that the identity is correct, our confidence is at least CHAPTER 1 Events and probability 53 1.1 Verifying Polynomial Identities 1.2 Axioms of Probability 1.3 Verifying Matrix Multiplication 1.4 A Randomized Min-Cut Algorithm