Download Computability - Homepages | The University of Aberdeen

CS4018
Formal Models of Computation
weeks 20-23
Computability and Complexity
Kees van Deemter
(partly based on lecture notes by Dirk Nikodem)
First set of slides:
Introduction to Computability
•
•
•
•
Introduction: computability
Countably and uncountably infinite sets
Enumeration and diagonalisation
Some functions must be uncomputable
The un-motto of these lectures:
• “Put the right kind of software into a
computer and it will do whatever you want
it to do. There may be limits on what you
can do with the machines themselves, but
there are no limits on what you can do with
software.” (TIME Magazine, 1984. Quoted with
abhorrence by Harel, The Science of Computing)
• Goal of these lectures: sharpen your
feeling for the limits on (1) what computers
can achieve at all, and (2) what they can
achieve given certain constraints on time
and space.
• Side effect: introduce you to some
interesting mathematics. (Includes two
famous open questions.)
Lectures week 20,21
1. Computability: Some problems can be
solved by computer, others cannot. Why
not?
2. Complexity:
– Some programs are faster than others. How
can one make this notion precise?
– Some problems require programs that take
a long time running. How can we tell?
• Four weeks is a short time: Computability
alone could take up an entire course.
• This necessitates an informal approach,
using as little formalism as possible.
(Proofs not based on Turing machines
etc.)
• Key insights can still be gained in this way
1. Book Recommendation
• Algorithmics: the Spirit of Computing,
by David Harel
• The Science of Computing,
by David Harel
• Discrete Mathematics and its Applications,
by Kenneth Rosen
2,3,4 What is computability?
• `Computation’ is a pre-theoretical notion,
The Collins Dictionary and Thesaurus
(1987) reflects an outdated view:
– “computation: a calculation involving
numbers or quantities.”
– “calculate: to solve by a mathematical
procedure”
• computing scientists have turned it into a
technical term
Informal explication of `computation’
• Suppose you want to achieve some goal
• Suppose you have a procedure for doing this
(stated in some objective way)
• Procedure relies on primitives that are
unproblematic
What makes …
• a good cooking recipe?
• good instructions for using data projector?
Desirable: (e.g., think of long division)
• Statement of procedure is explicit, clear
and unambiguous; no inventiveness
required
• Statement is finite, and requires finite
resources (e.g., money, memory)
• The outcome of the procedure can be
trusted
This is called an effective procedure,
a computation, or an algorithm.
History
• Gottlob Leibnitz (1700) “Calculemus”: let’s
decide all disputes by computation
• David Hilbert (1900): “Can all questions
of mathematics be answered algorithmically?”
• Kurt Goedel (1931): This is not possible, even
for number theory. (Result was independent of
the notion of computation.)
• Alan Turing (1936): The first formal model of
computation, the Turing Machine
Other formalisations:
– Lambda calculus,
– Recursive functions
Church’s thesis
Proven equivalent:
– Turing machines
– Lambda calculus
– Recursive functions
– Any general-purpose programming language
Thesis: this is all there is to computation
Some variations are still possible
How about another requirement:
“Goal is always achieved in finite time” (?)
One answer: termination
is always necessary
This position has the advantage of clarity.
Let us look at an example.
Here is a simple program for determining
whether a number n is prime:
(in quasi-PASCAL, as in lecture notes)
Problem: is n a prime number?
program prime (input, output);
var n, i : integer;
begin
read (n);
i := 2;
while (i<n) and (n mod i <> 0)
{n is not divisible by i}
do i := i+1;
if i=n then writeln(“n is prime”)
else writeln(“n is not prime”);
end
• This procedure is slow, but we will not
worry about speed for now
(until week 22: complexity)
• Program terminates. If it did not, it would
not be useful. Note: it addresses a
(decision) problem: a decision YES/NO
• What if the goal of the program is to find
something? What if that something may
not exist?
• Example: find the smallest prime number
that contains a string of n sevens.
“Find prime containing
string of n sevens”
• One approach: Set n, then run previous
procedure with increasingly large
numbers. For each prime, test whether it
contains a string of n sevens, until you
have found one.
• For some n, you may never find one.
Does this make the procedure noneffective? Is it not a computation?
Some people have answered NO:
(second response)
“Decision problems require termination.
Other procedures do not.”
Both responses are legitimate:
The meaning of computation (effective
procedure, algorithm) is not cast in stone
Another example (Harel)
• Algorithm A:
(1) while x<>1 do x:=x-2;
(2) stop
• Clearly, A terminates iff x is odd
Another example (Harel)
• Algorithm B:
(1) while x<>1 do
if x is even
then do x:=x/2;
else do x:=3x+1
(2) stop
• Try it out with inputs
x=3, x=7
(1) while x<>1 do
if x is even
then do x:=x/2;
else do x:=3x+1
(2) stop
• x=3 trace:
x:=10,5,16,8,4,2,1
• x=7 trace:
x:=22,11,34,17,52,26,13,40,20,10,5,16,8,4,2,1
• B terminates for any known input, but no
proof was available as to whether B must
always terminate.
• The strict view implies that we don’t know
whether B is an algorithm.
(We won’t take a stand here)
Questions about algorithms
• Does the algorithm terminate?
[More about this below]
• How long does it take? [Weeks 22,23]
• How much memory does it use?
• Does the algorithm always achieve the
intended goal? [Correctness: other
courses]
Proof methods
• Enumeration
• Diagonalisation
• Reduction
Enumeration
Structure of proof:
there exist n algorithms
there exist m problems
m>n
--------------------------------- therefore
some problems don’t have an algorithm
• Complication: the relevant numbers are
large
• Is it meaningful to ask how many
programs there are? - YES
• Georg Cantor (1845-1918) invented the
calculus of infinite numbers.
• Initially controversial, now very influential
• Cantor’s starting point: N = the set of
natural numbers = {0,1,2,…}. How many
elements? Not 0, not 1, not 2, …
• Cantor called this number 0
(the set is countably infinite).
• The next number is called 1
the next one 2 and so on.
(Together: uncountably infinite)
• Question: what is 1 ?
Cantor’s key principle:
“Sets A and B have the same cardinality iff
there exists a 1-1 correspondence between
them”. (Also called bijection)
Let A=N. Then how about
1. B={1,2,3,4,…}?
2. B={1,3,5,7,…}?
1. B={1,2,3,4,…}? Yes, use
f:AB such that f(x)=x+1
A: 0 1 2 3 4 5
B: 1 2 3 4 5 6
f is a function, since
xA yB(f(x,y))
f(x,y) & f(x,z) implies y=z
f is an injection, since
f(x,y) & f(z,y) implies x=z
f is a surjection, since
yB xA(f(x,y))
2. B={1,3,5,7,…}? Yes, B has the same
cardinality as N. To see this, use
f:AB such that f(x)=2x+1
A: 0
B: 1
1
3
2
5
3
7
4
9
5
11
Note that a 1-1 correspondence with N
is an enumeration.
• So, many sets are equally large as N.
They all have cardinality 0
• One could easily think that all infinite
sets have this cardinality: there would not
exist a cardinality 1
• This turns out to be false. In fact, the real
numbers are a larger set than N.
– (In fact, for all sets A, the power set of A is
larger than A)
Key observation: |||| > ||N||
Important for two reasons:
1. We need this fact for our proof by
enumeration
2. Proof of |||| > ||N|| is seminal
Proof proceeds by Reductio Ad Absurdum:
By deducing a contradiction from p,
we prove that ¬p
Here goes:
• Suppose  was countably infinite
(i.e., its cardinality is 0)
• Then we could completely enumerate its
elements:
a1. (first real number)
a2. (second real number)
a3. (etc.)
• In decimal notation, real numbers have
0 digits:
a1 = <a11 , a12 , a13 ….>
a2 = <a21 , a22 , a23 ….>
a3 = <a31 , a32 , a33 ….>
Now construct a number n(ew)=def
<a11 +1 ,a22 +1 ,a33 +1, …>
n differs from the i-th number on the i-th digit
Which of the enumerated numbers is n?
• n is not a1 because of a11
• n is not a2 because of a22 Etc.
So, n is not in the list! Consequently,
the enumeration was not complete after all
• We have reached a contradiction:
the enumeration is complete and
incomplete
• Adding the number n does not help …
• (Homework: does the argument
apply to the set Q of all fractions?)
• Diagonalisation is a very productive proof
method in computability
Now back to the first proof method:
Enumeration
• Enumeration is much simpler than
diagonalisation. Two questions to ask:
(1) How many computations are there?
(2) How many things are there to compute?
• Regarding (1): Let’s focus on programs
• Regarding (2): Let’s focus on numerical
functions: from integers to integers
(called integer functions)
Enumeration (1)
• Regarding simplification (1):
“computations ~ programs”
• This is a nontrivial step. In computability,
people are usually more careful: they work
with a formalism that is as simple as
possible, while still allowing every
reasonable `computation’: e.g.,
Turing machines.
• Church’s thesis is not an exact result, yet it
is practically universally believed
• Relevance for us: if we prove something
about what can be computed using
PASCAL or JAVA, then we assume it to
hold for any programming language that
anyone could think of
• Hence the simplification
“computations ~ programs”
Enumeration (2)
Regarding simplification (2):
things to calculate ~integer functions
This is unproblematic. (If some integer
functions are uncomputable then some
problems are uncomputable)
Noncomputability proof by
enumeration:
(1) The number of programs is 0.
(2) The number of integer functions is 1
(3) Therefore some integer functions
cannot be programmed
(4) Therefore some integer functions
are uncomputable [by Church’s thesis]
Noncomputability proof by
enumeration:
(1) The number of programs is 0. Proof:
- The number of programs whose
length is 1 character can be enumerated
- The number of programs whose
length is 2 characters can be
enumerated. Etc.
- Arrange these enumerations head to tail,
and the result is another enumeration
Noncomputability proof by
enumeration:
(2) The number of integer functions is 1
Proof: by diagonalisation: Suppose there
existed an enumeration of integer functions:
F1, F2, F3, … . Then construct a function
N(ew) using the following specification:
• N differs from F1 in its value for arg. 1
• N differs from F2 in its value for arg. 2
• Etc.
More concretely, we might define
• N(1)=F1(1)+1
• N(2)=F2(2)+1
• N(3)=F3(3)+1
• Etc.
• It follows that not all integer functions are
computable
• (Note that the argument can be applied to
non-integer functions if we number their
arguments and values.)
Back to Cantor
• Cantor proved that |||| > ||N||
• The proof does not show that nothing may lie in
between. The Continuum Problem:
– “Is any set larger than N but smaller than ?”
– Equivalent: is |||| = 1 ?
• One of the great open questions of pure
mathematics. Nothing computational appears to
hinge on it
• This is not true for the other open problem that
we shall meet (week 23)