Download Probabilistic Algorithms

Complexity
22-1
Probabilistic Algorithms
Complexity
Andrei Bulatov
Complexity
Non-Deterministic vs. Probabilistic
All algorithms we nave seen so far are either deterministic or impractical
(non-deterministic)
To make non-deterministic algorithms more practical we introduce
probabilistic algorithms
A probabilistic algorithm (Turing Machine) is a non-deterministic algorithm
that makes non-deterministic choices randomly, e.g. by flipping a coin
This is still not practical, because sometimes the algorithm should be
extremely lucky to solve problems
22-2
Complexity
22-3
Interactive Proofs
Prover
Verifier
Has unlimited computational
power
Can perform polynomial time
computations
Wants to convince Verifier
in something
Accepts or rejects after
performing some computation
They can exchange massages
Complexity
Proofs for Problems in NP
SAT
Prover and Verifier get an instance of SAT
Prover solves the instance using his unlimited computational power
and send a satisfying assignment to Verifier
Verifier checks (in polynomial time) if what obtained is a satisfying
assignment, and accepts if it is or rejects otherwise
22-4
Complexity
Problems from coNP
Graph Non-Isomorphism
Instance: Graphs G and H.
Question: Are G and H isomorphic?
This problem belongs to coNP, but is not believed to be coNP-complete
Apparently, there is no way to prove interactively that two graphs are
not isomorphic
22-5
Complexity
Randomized Verifier
Now suppose that Verifier has a fair coin
Given graphs G and H
• Verifier choose one of G and H by flipping a coin
• Verifier then rename somehow the vertices of the chosen
graph and send it to Prover
• Prover decides which graph it received
• Prover send the answer to Verifier
• Repeat the procedure
22-6
Complexity
Analysis
If the graphs are not isomorphic then Prover always gives the right
answer
If the graphs are isomorphic then Prover gives a correct answer with
probability 1/2
Therefore if Prover is wrong we conclude that the graphs are isomorphic
If after n repetitions of the protocol, Prover gives only right answers,
1
then Verifier can conclude with probability n that graphs are not
2
isomorphic
22-7
Complexity
Probabilistic Turing Machines
Definition
A Probabilistic Turing Machine is a nondeterministic polynomial
time Turing Machine PT such that
• from each configuration of PT, there are at most two
possible next configurations
• PT chooses which of the two possible next configurations
to take by flipping a fair coin
Thus, with each computational path, we can associate the probability of
taking this path. This probability is equal to 21k where k is the number
of coin flips made along this path
Denote this probability by Pr[p]
22-8
Complexity




22-9

Define the probability that PT accepts w to be
Pr[ PT accepts w] 
 Pr[ p]
p is an
accepting path
Clearly Pr[ PT rejects w]  1  Pr[ PT accepts w]


Complexity
Class BPP
Definition
A Probabilistic Turing Machine PT recognizes language L with
error probability  if
• w  L implies Pr[PT accepts w]  1 – 
• w  L implies Pr[PT rejects w]  1 – 
We say that PT operates with error probability 2  f ( n )
if the above inequalities hold for   2  f ( n )
where n is the
length of w
Definition
BPP is the class of languages that are recognizable by
probabilistic Turing Machines with error probability of 1/3
22-10
Complexity
Amplification
The error probability 1/3 may seem random
Actually, we can choose any value 0    1
Amplification Lemma
Let 0    1. Then for any polynomial p(n) and a
probabilistic TM PT1 that operates with error probability ,
there is a probabilistic TM PT2 that operates with an error
probability 2 p ( n )
The main idea is to run PT1 many times and then output the majority
of votes
22-11
Complexity
Math Prerequisites
Let X 1 , X 2 ,, X n be a series of independent experiments (for
example, coin flips) such that the probability of success in each of them
is p
Theorem (Chernoff Bound)
If p  1   for some   , then the probability that the
2
number of successes in a series of n experiments is less than
n is at most
e
2

 2n
6
22-12
Complexity
Proof of Amplification Lemma
Machine PT2 works as follows
On input w
• for i = 1 to t(|w|) do
- simulate PT1 on w
• if most runs of PT1 accept, then accept; otherwise reject
22-13
Complexity
22-14
Analysis
The number t(|w|) must be such that
e


 2t (|w|)
6
 2t (| w |)
6
 2  p (|w|)
  p(| w |) ln 2
 2 t (| w |)  6 p(| w |) ln 2
t (| w |) 
6 ln 2
2
p(| w |)
Complexity
22-15
Primes
Complexity
Andrei Bulatov
Complexity
22-16
The Problem
Primes
Instance: A positive integer k.
Question: Is k prime?
The complement of Primes, the Composite problem, belongs
to NP. Therefore Primes is in coNP
Recently M.Agarwal et al. Proved that Primes can be solved in polynomial
time
(see http://www.cse.iitk.ac.in/news/primality.html)
However, the probabilistic algorithm we are going describe is far more efficient
Complexity
22-17
Residues
For a positive integer n, we denote
•
Z n the set {0,1,2,…,n –1}
•
Z n the set {1,2,…,n – 1}
• ,, x y addition, multiplication and exponentiation modulo n
Z n together with these operations is called the set of residues modulo n
Every integer m, positive or negative, has a corresponding residue —
m mod n
For example,
17 mod 5 = 2
20 mod 5 = 0
-1 mod 5 = 4
Complexity
22-18
Complexity of Arithmetic
Given two integers, a and b, we can compute
• a + b in O(max(log a, log b))
• a  b in O(log a  log b)
a b cannot be computed in polynomial time, because the size of this
number is blog a
It is possible modulo n
Let b1b2 bk be the binary representation of b (k = log b)
Then b  b0 20  b1 21    bk 2k that implies
0
k
1
a b (mod n )  a b0 2  a b1 2   a bk 2
20
21
2k
First, we consecutively compute a , a ,, a in
Then we compute the product again in O ( k log 2 n )
Complexity
Prime and Coprime
Integers a and b are called coprime if their greatest common divisor is 1
For example, 16 and 27 are coprime, and 15 and 18 are not
Theorem (Chinese Remainder Theorem)
If p and q are coprime then, for any a and b, there is x
such that
x  a (mod p )
x  b(mod q)
For example, if p = 5, q = 3, and a = 2, b = 1, then x can be
chosen to be 7
22-19
Complexity
Fermat’s Theorem
Theorem (Fermat’s Little Theorem)
If p is prime then, for any a  Z p we have a p1  1(mod p )
If the converse were true, we could use it for a probabilistic primality test:
• Choose k residues modulo n;
• Compute their n –1 powers;
• Accept if all results are 1 (mod n), reject otherwise
22-20
Complexity
Carmichael Numbers
Unfortunately, the converse is true just “almost”
Definition
A number n passes Fermat’s test if a p1  1(mod p ) for all a
coprime with n
A number that passes Fermat’s test is called pseudo-prime

One can straightforwardly check that, for any a  Z561
, coprime with 561,
a 560  1(mod 561)
561 is a Carmichael number
n is said to be a Carmichael number if, for any prime divisor p of n,
p –1 | n – 1
Pseudo-prime = Prime + Carmichael
22-21
Complexity
Roots of 1
A square root of 1 modulo n is a number a such that a 2  1(mod n )
Clearly, 1 and -1 (that is n – 1) are always roots of 1, but if n is
composite, then it may have more than two roots of 1
For example,
8 has four roots of 1: 1, -1, 3, and 5
561 has eight: 1, -1, 188, 373 (find the remaining four)
Lemma
Any Carmichael number has at least 8 roots of 1
22-22
Complexity
22-23
Algorithm
On input n
• if n is even, then if n = 2 accept, otherwise reject
• select randomly a1 , a2 ,, ak  Z n
• for i = 1 to k do
- if ain1  1(mod n) then reject
- let n – 1 = st where s is odd and t  2 h is a power of 2
s20
i
- compute the sequence a
s2 j
i
- if a
s21
i
,a
s2h
i
,, a
 1 then
let j be the maximal with this property
if ais2
• accept
j 1
 1 then reject
modulo n
Complexity
Analysis
First we show that the algorithm does not give false negatives, that is
it accepts all prime numbers
If n = 2 then n is accepted. Let n be an odd prime number
Then n passes Fermat test
n cannot be rejected in the last line, because n has only two roots of 1
22-24
Complexity
22-25
Next we show that if n is composite, then Pr[n accepted]  2k
A number a  Z n such that a does not pass either Fermat test or the
square root test, is called a witness
It is enough to prove that Pr[a is a witness]  1/2, or, in other words,
that at least half of the elements of Z n are witnesses
For every nonwitness d we find a witness d´ such that if d1  d 2
then d '1  d ' 2
s20
For a nonwitness a the sequence a , a
1s only, or it contains -1 followed by 1s
s21
, , a
s2h
either contains
Nonwitnesses of both types are present: 1 is a nonwitness of the first
type, and -1 is a nonwitness of the second type
Complexity
22-26
Let d be a nonwitness of the second type such that the –1 appears in the
largest position in the sequence
Let d s2  1 and d s2
j
j 1
1
Since n is composite, n = qr for some coprime q and r
Note that
1  1(mod q)
1  1(mod r )
and
 1  1(mod q)
 1  1(mod r )
By the Chinese Reminder Theorem, there is t such that
t  d (mod q)
t  1(mod r )
s2 j
t  1(mod q)
s2 j
t  1(mod r )
therefore
Hence t is a witness, because t s2  1(mod p ) but t s2
j
j 1
 1(mod p )
Complexity
22-26
Now, for every nonwitness a we set a´ = a · t
• a´ is a witness, because a s2  1(mod p ) and
j
(a' )
(a' )
s2 j
 ( at )
s2 j 1
s2 j
 ( at )
 t
s2 j 1
s2 j
t
 1(mod p )
s2 j 1
but
 1(mod p )
• if a1  a2 then a '1  a ' 2
Assume the contrary
a '1  a ' 2 (mod p )
ta1  ta2 (mod p )
Then, since t s2
j 1
 1(mod p ) we have t s2
Finally, we have
a1  t s2
j 1
1
 t  a1  t s2
j 1
1
 t  a2  a2
j 1
1
 t  1(mod p )