Download Algorithms, Nondeterminism, Complexity, and Randomicity

Algorithms,
Nondeterminism,
Complexity, and
Randomicity
Alberto Pettorossi
Department of Informatics, Systems, and Production,
University of Roma Tor Vergata
4 March 2009
EXISTENCE OF NON-COMPUTABLE FUNCTIONS
N: natural numbers
℘(N): set of all subsets of natural numbers
N → N: functions from N to N
Progs: programs in C++ (or Java)
Theorem. | N | = | Progs |
Theorem. | N → N | = | ℘(N) | = | R |
Cantor Theorem. For any set A, there is no bijection from A to ℘(A).
Corollary. There is a function f from N to N such that it has no corresponding C++ program, i.e., f is a non-computable function.
Theorem. | R − N | = | R |.
Corollary. (i) There are | N | computable functions from N to N.
(ii) There are | R | (computable or non-computable) functions from N to N.
2
VARIOUS LEVELS OF COMPUTABILITY
(above) : non-computable functions
(type 0) (r.e.) : Turing Machines (TM) = FA + 2 stacks (push x, pop, is-empty ?)
computable functions
(rec) : Turing Machines which terminate for all inputs
total computable functions
(type 1) : . . .
(type 2) : Pushdown Automata = FA + 1 stack (push x, pop, is-empty?)
{an bn | n ≥ 1}
(type 3) : Finite Automata (= FA)
0
A
0
1
1
B
1
1
C
0
0
D
(divisibility by 3 for binary numbers)
1100 (= 12) takes from A to B.
Note 1.
FA + 2 stacks
(push x, pop, is-empty?)
= FA + 2 counters (+1, −1, is-0 ?)
(read, write, move L, move R)
= FA + 1 tape
Note 2. Also the encoding of the input and the decoding of the output should
be computable (for instance, the encoding of 12 into 1100).
3
A TURING MACHINE
A Turing Machine is a septuple hQ, Σ, Γ, q0 , B, F, δi, where:
- Q is the set of states,
- Σ is the input alphabet,
- Γ is the tape alphabet,
- q0 is the initial state,
- B is the blank symbol,
- F is the set of the final states, and
- δ is a partial function from Q × Γ to Q × (Γ − {B}) × {L, R}, called the
transition function, which defines the set of instructions or quintuples of the
Turing Machine.
We assume that Q and Γ are disjoint sets, Σ ⊆ Γ−{B}, q0 ∈ Q, B ∈ Γ, and
F ⊆ Q.
finite automaton FA in state q
Σ = {a, b, c, d}
states: Q
initial state: q0
Γ = {a, b, c, d, B}
final states: F
- (the tape head moves
to the left and to the right)
?
b b a a b d B B B ...
α1
α2
A Turing Machine in the configuration b b a q a b d. The cells of the tape are
c1 , c2, . . . The head scans the cell c4 and reads the symbol a.
A Turing Machine accepts an input word w when starting from the configuration q0 w reaches a configuration w1 q w2 where q ∈ F .
For any expression e, by peq we denote the encoding of e.
A Turing Machine computes the function f : N → N iff for all n ∈ N, starting
from the configuration q0 pnq reaches a configuration q pf (n)q where q ∈ F .
4
PROBLEMS AS SETS OF WORDS
non-halting: { pxq $ pMq | Turing Machine M does not halt for input x}
⊆ (0 + 1)∗ $ (0 + 1)∗
halting:
{ pxq $ pMq | Turing Machine M halts for input x}
⊆ (0 + 1)∗ $ (0 + 1)∗
parsing:
{ pwq $ pGq | grammar G generates w}
⊆ (0 + 1)∗ $ (0 + 1)∗
...
⊆ 01∗
primes:
{01n | prime(n)}
sorting:
{x1# . . . #xn $ xi1 # . . . #xin | xi1 ≤ . . . ≤ xin }
⊆ (N#)n−1N $ (N#)n−1N
sum:
{01n01m01m+n0 | n, m ≥ 0}
...
5
⊆ 01∗01∗01∗0
RECURSIVELY ENUMERABLE SETS AND RECURSIVE SETS
• Non-recursively enumerable sets (subsets of L).
Non-semi-solvable (or non-semi-decidable) problems:
non-halting, . . .
• Recursively enumerable sets (subsets of L).
A set A ⊆ L is r.e. iff there exists a Turing Machine M such that
for all a ∈ A, M stops in a final state for the input a.
Semi-solvable (or semi-decidable) problems:
halting,
parsing for type 0 grammars, . . .
• Recursive sets (subsets of L).
A set A ⊆ L is rec iff there exists a Turing Machine M such that
- for all x ∈ L, M stops and
- for all a ∈ A, M stops in a final state for the input a.
Solvable (or decidable) problems:
parsing for type i grammars (with i = 1, 2, 3),
primes,
sorting,
sum, . . .
6
ALGORITHMS DENOTE R.E. SETS
Theorem. Given a Turing-complete programming language L, the interpreter I for the language L (maybe written in L) should allow non terminating executions.
2
Thus, there exist (i) a program P written in L, and (ii) an input x for P
such that the interpreter I, running on P and x, does not terminate.
Thus, if we want to consider I as an algorithm, then the class of algorithms
should denote r.e. sets.
Note. An algorithm is a Turing Machine.
The execution of an algorithm is not a finite sequence of well-defined steps,
as indicated in some books (see below), because: (i) the execution may not
terminate, (ii) the execution may be nondeterministic, and (iii) the notion of
step is unclear.
...
Input - L’algoritmo deve avere un input contenuto in un insieme definito I.
Output - Da ogni insieme di valori in input, l’algoritmo produce un insieme
di valori in uscita che comprende la soluzione.
Determinatezza - I passi dell’algoritmo devono essere definiti precisamente.
Finitezza - Un algoritmo deve produrre la soluzione in un numero di passi
finito (eventualmente molto grande) per ogni possibile input definito su I.
Efficacia - Deve essere possibile effettuare ogni passo del l’algoritmo esattamente ed in un tempo finito.
Generalità - L’algoritmo deve essere valido per ogni insieme di dati contenuti
in I e non solo per alcuni.
...
7
DETERMINISM AND NONDETERMINISM
Deterministic acceptance:
starting from the initial state, the computation of the automaton stops in a
final state.
Nondeterministic acceptance:
there exists a computation starting from the initial state such that the automaton stops in a final state.
(type 0) : Nondeterministic Turing Machines = Deterministic Turing Machines
(type 2) : Nondeterministic Pushdown Automata
⊃ Deterministic Pushdown Automata
(type 3) : Nondeterministic Finite Automata = Deterministic Finite Automata
S
a
a
A1
b
A2
c
B
=
C
8
S
a
A
b
c
B
C
TIME COMPLEXITY
• Polynomial relationship between the time complexity of Random Access
Machines (or C++ programs) and deterministic Turing Machines.
There exists k ≥ 0 such that for every algorithm A from N to N,
there exists a Random Access Machine (without multiplication or division)
which takes O(n) time units for executing A iff
there exists a deterministic Turing Machine which takes O(nk ) time units
(that is, polynomial time w.r.t. n) for executing A.
The same for space, instead of time.
• P : Deterministic Polynomial Class.
A predicate λd. p(d) : D → {true, false} is in P iff there exists a deterministic
Turing Machine M such that for all d ∈ D,
M evaluates p(d) in polynomial time w.r.t. size(d).
One can show that the class P is closed w.r.t. negation, that is, P = co-P.
Thus, in the above definition of the class P, actually, it does not matter
whether the evaluation of p(d) returns true or false.
9
THE NP CLASS
• NP : Nondeterministic Polynomial Class.
A predicate λd. p(d) : D → {true, false} is in NP iff
(i) there exists a deterministic Turing Machine M,
(ii) there exists a finite alphabet Σ,
(iii) there exists a predicate π : D×Σ∗ → {true, false}, and
(iv) there exists k ≥ 0 such that
- for all d ∈ D, M evaluates p(d) by returning the value: ∃w ∈ W. π(d, w)
where W = {w ∈ Σ∗ | size(w) = (size(d))k }, and
- for all d ∈ D, w ∈ Σ∗, M evaluates π(d, w) in polynomial time
w.r.t. size(d)×size(w).
The evaluation of p(d) in NP is a search, in a polynomially deep tree, of a
leaf, if any, associated with a word w such that π(d, w) holds.
u
J
J
0
J1
J
u
AA
A
AA
A
0 AA1
0 A1
A Au A
ee
ee
...
...
e
eeu
B
B
B
B
B
B
B
B
B ... 0 B 1 B
B
B
B
B B v
6
depth of
the order of
(size(d))k
?
w = 01 . . . 0
(In the picture we have assumed Σ = {0, 1}.)
• It is an open problem whether or not NP is closed w.r.t. negation, that is,
whether or not NP = co-NP.
• NP ∩ co-NP 6= ∅.
PRIME and CO-PRIME both belong to NP and co-NP (see below).
• If P = NP then NP = co-NP (because P = co-P).
The reversed implication, if NP = co-NP then P = NP, does not hold.
10
CO-PRIME IS IN NP
CO-PRIME
Input: a positive integer p in binary notation, using k = ⌈log2 p⌉ bits.
Output: ‘yes’ iff p is not a prime number.
An instance of the CO-PRIME problem can be solved as follows:
Step 1. We construct the set W ∈ {0, 1}k of the encodings of all positive
numbers i, for 1 < i < p.
Step 2. Then we return ‘yes’ iff there exists an encoding w ∈ W such that w
represents a number which divides p.
2
(Obviously, to test whether or not a given number divides p requires polynomial time with respect to k.)
As recently shown, actually the CO-PRIME problem is in P, because the
PRIME problem (see below) is in P.
11
PRIME IS IN NP
(1)
PRIME
Input: a positive integer p in binary notation, using k = ⌈log2 p⌉ bits.
Output: ‘yes’ iff p is a prime number.
• The proof that a problem is in NP is, in general, not a proof that the
negated problem is in NP.
Theorem 1. [Fermat Theorem] A number p (> 2) is prime iff there exists x
with 1 < x < p, such that
(α): xp−1 = 1 (mod p) and
(β): for all i, with 1 < i < p−1, xi 6= 1 (mod p).
Example. We have that p = 7 is a prime number because there exists x,
which is 3, such that 1 < x < 7 and
(α) holds because 37−1 = 729 = 1 (mod 7), and
(β) holds because
32 = 2 (mod 7)
(indeed, 3 × 3 = 9 = 7+2)
3
3 = 6 (mod 7)
(indeed, 2 × 3 = 6)
34 = 4 (mod 7)
(indeed, 6 × 3 = 18 = (2 × 7)+4)
5
2
3 = 5 (mod 7)
(indeed, 4 × 3 = 12 = 7+5)
Theorem 2. Assume that: (i) p > 2, (ii) 1 < x < p, and (iii) xp−1 = 1 (mod p).
If for all pj ∈ primefactors(p−1), x(p−1)/pj 6= 1 (mod p)
then for all i, with 1 < i < p−1, we have that xi 6= 1 (mod p).
12
PRIME IS IN NP
(2)
Theorem 2. Assume that: (i) p > 2, (ii) 1 < x < p, and (iii) xp−1 = 1 (mod p).
If for all pj ∈ primefactors(p−1), x(p−1)/pj 6= 1 (mod p)
then for all i, with 1 < i < p−1, we have that xi 6= 1 (mod p).
• The size of the input p is ⌈log2 p⌉.
• | {i | 1 < i < p−1} | (= p−3) tests are too many, because p−3 = O(2log2 p ).
• | primefactors(p−1) | tests are not too many,
because | primefactors(p−1) | ≤ ⌊log2 p⌋.
Example (continued). In order to test Condition (β) we need to test that:
32 6= 1 (mod 7)
(∗)
3
3 6= 1 (mod 7)
(∗)
4
3 6= 1 (mod 7)
35 6= 1 (mod 7).
Indeed, all these inequalities hold, because
32 = 9 = 2 (mod 7)
33 = 27 = 6 (mod 7)
34 = 81 = 4 (mod 7)
35 = 243 = 5 (mod 7).
By Theorem 2, since primefactors(7−1) = {2, 3},
in order to test (β), it is enough to test only the disequations marked with (∗).
Indeed, we have that:
36/2 6= 1 (mod 7) and
2
36/3 6= 1 (mod 7).
13
FROM P TO EXPTIME
=
⊆
EXPTIME Simplex Algorithm for linear programming
min z = cT x with Ax = b, x ≥ 0
( ? open problem)
(= NSPACE) membership of context-sensitive languages
⊆
PSPACE
=
( ? open problem)
satisfiability of propositional formulas in CNF
⊆
NP
=
( ? open problem)
emptiness of context-free languages
’s is
, because P ⊂ EXPTIME.
⊂
One of the
⊆
P
14
INTERACTIVE PROOF SYSTEM
Prover:
TM Alice
• input tape:
pxq
↔
• random tape: 0101011011 . . .
→
• working tape: . . .
↔
(no limitations)
x ∈ L ? Verifier:
(1)
TM Bob
• input tape:
pxq
↔
• random tape: 1001010110 . . .
→
• working tape: . . .
↔
total polynomial time w.r.t. size(x)
• interaction tape: m0 | m1 | m2 | . . .
→
In the interation tape:
- mutual exclusive use by TM Alice and TM Bob according to a protocol.
- every message mi uses polynomial space.
Proof in probability:
2
- if x ∈ L then Bob accepts x with probability ≥ .
3
1
- if x 6∈ L then Bob accepts x with probability ≤ .
3
Definition. The class IP is the class of the languages L recognized by an
interactive proof system.
2
1
1
1
Note. We can replace the values and by any other values > and < ,
3
3
2
2
respectively, without changing the definition of the class IP.
Indeed, by repeating the protocol we can reach any desired level of probability.
15
INTERACTIVE PROOF SYSTEM
(2)
Definition. The class IP is the class of languages recognized by an interactive proof system.
Fact. NP ⊆ IP.
Proof. Here is the protocol. The TM Alice sends to the TM Bob the
encoding of the TM Alice and the list of the choices which are made by
the TM Alice for accepting the input string x. Then TM Bob in polynomial time will act according to that list of choices and will accept x (with
probability 1).
2
Theorem. IP = PSPACE.
16
INTERACTIVE PROOF SYSTEM
(3)
GRAPH NON-ISOMORPHISM.
Input: two directed or undirected graphs G1 and G2 .
Output: ‘yes’ iff the graphs G1 and G2 are non-isomorphic (that is, they are
not equal modulo a permutation of the vertices).
Fact. The GRAPH NON-ISOMORPHISM problem is in IP.
Proof. Here is the protocol. G1 and G2 , are given both to Alice and Bob.
The protocol has two steps.
Step 1.
(1.1) Bob randomly chooses an i in {1, 2}.
(1.2) Bob sends to Alice a graph H which is isomorphic to Gi and is obtained
by Bob by randomly permuting the vertices of Gi .
(1.3) Alice looks for a j in {1, 2} such that graph Gj is isomorphic to H
and she sends j back to Bob. (Note that Alice will always succeed in
finding j, but if the two graphs G1 and G2 are isomorphic, j may be
different from i.)
Step 2. It is equal to Step 1.
When Bob receives the second value of j, he announces the end of the
protocol.
If the two given graphs are non-isomorphic, in both steps Alice returns to
Bob the same value he sent to her, and Bob accepts the given graphs as
non-isomorphic with probability 1.
If the two given graphs are isomorphic, since in each step the probability
1
that Alice guesses the i chosen by Bob is , we have that the probability
2
1
that Bob accepts the given graphs as non-isomorphic is at most (which
4
1
is ≤ ).
2
3
17
INTERACTIVE PROOF SYSTEM
(4)
GRAPH ISOMORPHISM.
Input: two directed or undirected graphs G1 and G2 .
Output: ‘yes’ iff the graphs G1 and G2 are isomorphic (that is, they are equal
modulo a permutation of the names of their vertices).
Fact 1. The GRAPH ISOMORPHISM problem is in NP.
Fact 2. The GRAPH NON-ISOMORPHISM problem is in co-NP.
It is an open problem whether or not the GRAPH NON-ISOMORPHISM
problem is in NP.
18
References
[1] J. E. Hopcroft and J. D. Ullman. Introduction to Automata Theory,
Languages and Computation. Addison-Wesley, 1979.
[2] E. Mendelson. Introduction to Mathematical Logic. Wadsworth &
Brooks/Cole Advanced Books & Software, Monterey, California, Usa,
Monterey, California, Usa, 1987. Third Edition.
[3] A. Pettorossi. Elements of Computability, Decidability, and Complexity.
Aracne Editrice, Roma, Italy, 2007. Second Edition.
19