Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Complexity 22-1 Probabilistic Algorithms Complexity Andrei Bulatov Complexity Non-Deterministic vs. Probabilistic All algorithms we nave seen so far are either deterministic or impractical (non-deterministic) To make non-deterministic algorithms more practical we introduce probabilistic algorithms A probabilistic algorithm (Turing Machine) is a non-deterministic algorithm that makes non-deterministic choices randomly, e.g. by flipping a coin This is still not practical, because sometimes the algorithm should be extremely lucky to solve problems 22-2 Complexity 22-3 Interactive Proofs Prover Verifier Has unlimited computational power Can perform polynomial time computations Wants to convince Verifier in something Accepts or rejects after performing some computation They can exchange massages Complexity Proofs for Problems in NP SAT Prover and Verifier get an instance of SAT Prover solves the instance using his unlimited computational power and send a satisfying assignment to Verifier Verifier checks (in polynomial time) if what obtained is a satisfying assignment, and accepts if it is or rejects otherwise 22-4 Complexity Problems from coNP Graph Non-Isomorphism Instance: Graphs G and H. Question: Are G and H isomorphic? This problem belongs to coNP, but is not believed to be coNP-complete Apparently, there is no way to prove interactively that two graphs are not isomorphic 22-5 Complexity Randomized Verifier Now suppose that Verifier has a fair coin Given graphs G and H • Verifier choose one of G and H by flipping a coin • Verifier then rename somehow the vertices of the chosen graph and send it to Prover • Prover decides which graph it received • Prover send the answer to Verifier • Repeat the procedure 22-6 Complexity Analysis If the graphs are not isomorphic then Prover always gives the right answer If the graphs are isomorphic then Prover gives a correct answer with probability 1/2 Therefore if Prover is wrong we conclude that the graphs are isomorphic If after n repetitions of the protocol, Prover gives only right answers, 1 then Verifier can conclude with probability n that graphs are not 2 isomorphic 22-7 Complexity Probabilistic Turing Machines Definition A Probabilistic Turing Machine is a nondeterministic polynomial time Turing Machine PT such that • from each configuration of PT, there are at most two possible next configurations • PT chooses which of the two possible next configurations to take by flipping a fair coin Thus, with each computational path, we can associate the probability of taking this path. This probability is equal to 21k where k is the number of coin flips made along this path Denote this probability by Pr[p] 22-8 Complexity 22-9 Define the probability that PT accepts w to be Pr[ PT accepts w] Pr[ p] p is an accepting path Clearly Pr[ PT rejects w] 1 Pr[ PT accepts w] Complexity Class BPP Definition A Probabilistic Turing Machine PT recognizes language L with error probability if • w L implies Pr[PT accepts w] 1 – • w L implies Pr[PT rejects w] 1 – We say that PT operates with error probability 2 f ( n ) if the above inequalities hold for 2 f ( n ) where n is the length of w Definition BPP is the class of languages that are recognizable by probabilistic Turing Machines with error probability of 1/3 22-10 Complexity Amplification The error probability 1/3 may seem random Actually, we can choose any value 0 1 Amplification Lemma Let 0 1. Then for any polynomial p(n) and a probabilistic TM PT1 that operates with error probability , there is a probabilistic TM PT2 that operates with an error probability 2 p ( n ) The main idea is to run PT1 many times and then output the majority of votes 22-11 Complexity Math Prerequisites Let X 1 , X 2 ,, X n be a series of independent experiments (for example, coin flips) such that the probability of success in each of them is p Theorem (Chernoff Bound) If p 1 for some , then the probability that the 2 number of successes in a series of n experiments is less than n is at most e 2 2n 6 22-12 Complexity Proof of Amplification Lemma Machine PT2 works as follows On input w • for i = 1 to t(|w|) do - simulate PT1 on w • if most runs of PT1 accept, then accept; otherwise reject 22-13 Complexity 22-14 Analysis The number t(|w|) must be such that e 2t (|w|) 6 2t (| w |) 6 2 p (|w|) p(| w |) ln 2 2 t (| w |) 6 p(| w |) ln 2 t (| w |) 6 ln 2 2 p(| w |) Complexity 22-15 Primes Complexity Andrei Bulatov Complexity 22-16 The Problem Primes Instance: A positive integer k. Question: Is k prime? The complement of Primes, the Composite problem, belongs to NP. Therefore Primes is in coNP Recently M.Agarwal et al. Proved that Primes can be solved in polynomial time (see http://www.cse.iitk.ac.in/news/primality.html) However, the probabilistic algorithm we are going describe is far more efficient Complexity 22-17 Residues For a positive integer n, we denote • Z n the set {0,1,2,…,n –1} • Z n the set {1,2,…,n – 1} • ,, x y addition, multiplication and exponentiation modulo n Z n together with these operations is called the set of residues modulo n Every integer m, positive or negative, has a corresponding residue — m mod n For example, 17 mod 5 = 2 20 mod 5 = 0 -1 mod 5 = 4 Complexity 22-18 Complexity of Arithmetic Given two integers, a and b, we can compute • a + b in O(max(log a, log b)) • a b in O(log a log b) a b cannot be computed in polynomial time, because the size of this number is blog a It is possible modulo n Let b1b2 bk be the binary representation of b (k = log b) Then b b0 20 b1 21 bk 2k that implies 0 k 1 a b (mod n ) a b0 2 a b1 2 a bk 2 20 21 2k First, we consecutively compute a , a ,, a in Then we compute the product again in O ( k log 2 n ) Complexity Prime and Coprime Integers a and b are called coprime if their greatest common divisor is 1 For example, 16 and 27 are coprime, and 15 and 18 are not Theorem (Chinese Remainder Theorem) If p and q are coprime then, for any a and b, there is x such that x a (mod p ) x b(mod q) For example, if p = 5, q = 3, and a = 2, b = 1, then x can be chosen to be 7 22-19 Complexity Fermat’s Theorem Theorem (Fermat’s Little Theorem) If p is prime then, for any a Z p we have a p1 1(mod p ) If the converse were true, we could use it for a probabilistic primality test: • Choose k residues modulo n; • Compute their n –1 powers; • Accept if all results are 1 (mod n), reject otherwise 22-20 Complexity Carmichael Numbers Unfortunately, the converse is true just “almost” Definition A number n passes Fermat’s test if a p1 1(mod p ) for all a coprime with n A number that passes Fermat’s test is called pseudo-prime One can straightforwardly check that, for any a Z561 , coprime with 561, a 560 1(mod 561) 561 is a Carmichael number n is said to be a Carmichael number if, for any prime divisor p of n, p –1 | n – 1 Pseudo-prime = Prime + Carmichael 22-21 Complexity Roots of 1 A square root of 1 modulo n is a number a such that a 2 1(mod n ) Clearly, 1 and -1 (that is n – 1) are always roots of 1, but if n is composite, then it may have more than two roots of 1 For example, 8 has four roots of 1: 1, -1, 3, and 5 561 has eight: 1, -1, 188, 373 (find the remaining four) Lemma Any Carmichael number has at least 8 roots of 1 22-22 Complexity 22-23 Algorithm On input n • if n is even, then if n = 2 accept, otherwise reject • select randomly a1 , a2 ,, ak Z n • for i = 1 to k do - if ain1 1(mod n) then reject - let n – 1 = st where s is odd and t 2 h is a power of 2 s20 i - compute the sequence a s2 j i - if a s21 i ,a s2h i ,, a 1 then let j be the maximal with this property if ais2 • accept j 1 1 then reject modulo n Complexity Analysis First we show that the algorithm does not give false negatives, that is it accepts all prime numbers If n = 2 then n is accepted. Let n be an odd prime number Then n passes Fermat test n cannot be rejected in the last line, because n has only two roots of 1 22-24 Complexity 22-25 Next we show that if n is composite, then Pr[n accepted] 2k A number a Z n such that a does not pass either Fermat test or the square root test, is called a witness It is enough to prove that Pr[a is a witness] 1/2, or, in other words, that at least half of the elements of Z n are witnesses For every nonwitness d we find a witness d´ such that if d1 d 2 then d '1 d ' 2 s20 For a nonwitness a the sequence a , a 1s only, or it contains -1 followed by 1s s21 , , a s2h either contains Nonwitnesses of both types are present: 1 is a nonwitness of the first type, and -1 is a nonwitness of the second type Complexity 22-26 Let d be a nonwitness of the second type such that the –1 appears in the largest position in the sequence Let d s2 1 and d s2 j j 1 1 Since n is composite, n = qr for some coprime q and r Note that 1 1(mod q) 1 1(mod r ) and 1 1(mod q) 1 1(mod r ) By the Chinese Reminder Theorem, there is t such that t d (mod q) t 1(mod r ) s2 j t 1(mod q) s2 j t 1(mod r ) therefore Hence t is a witness, because t s2 1(mod p ) but t s2 j j 1 1(mod p ) Complexity 22-26 Now, for every nonwitness a we set a´ = a · t • a´ is a witness, because a s2 1(mod p ) and j (a' ) (a' ) s2 j ( at ) s2 j 1 s2 j ( at ) t s2 j 1 s2 j t 1(mod p ) s2 j 1 but 1(mod p ) • if a1 a2 then a '1 a ' 2 Assume the contrary a '1 a ' 2 (mod p ) ta1 ta2 (mod p ) Then, since t s2 j 1 1(mod p ) we have t s2 Finally, we have a1 t s2 j 1 1 t a1 t s2 j 1 1 t a2 a2 j 1 1 t 1(mod p )