* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Survey

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts

Large numbers wikipedia, lookup

Wiles's proof of Fermat's Last Theorem wikipedia, lookup

Brouwer–Hilbert controversy wikipedia, lookup

Infinitesimal wikipedia, lookup

List of important publications in mathematics wikipedia, lookup

A New Kind of Science wikipedia, lookup

Foundations of mathematics wikipedia, lookup

Turing's proof wikipedia, lookup

Factorization of polynomials over finite fields wikipedia, lookup

Computability theory wikipedia, lookup

Mathematical proof wikipedia, lookup

Elementary mathematics wikipedia, lookup

Georg Cantor's first set theory article wikipedia, lookup

Halting problem wikipedia, lookup

Transcript

CS4018 Formal Models of Computation weeks 20-23 Computability and Complexity Kees van Deemter (partly based on lecture notes by Dirk Nikodem) First set of slides: Introduction to Computability • • • • Introduction: computability Countably and uncountably infinite sets Enumeration and diagonalisation Some functions must be uncomputable The un-motto of these lectures: • “Put the right kind of software into a computer and it will do whatever you want it to do. There may be limits on what you can do with the machines themselves, but there are no limits on what you can do with software.” (TIME Magazine, 1984. Quoted with abhorrence by Harel, The Science of Computing) • Goal of these lectures: sharpen your feeling for the limits on (1) what computers can achieve at all, and (2) what they can achieve given certain constraints on time and space. • Side effect: introduce you to some interesting mathematics. (Includes two famous open questions.) Lectures week 20,21 1. Computability: Some problems can be solved by computer, others cannot. Why not? 2. Complexity: – Some programs are faster than others. How can one make this notion precise? – Some problems require programs that take a long time running. How can we tell? • Four weeks is a short time: Computability alone could take up an entire course. • This necessitates an informal approach, using as little formalism as possible. (Proofs not based on Turing machines etc.) • Key insights can still be gained in this way 1. Book Recommendation • Algorithmics: the Spirit of Computing, by David Harel • The Science of Computing, by David Harel • Discrete Mathematics and its Applications, by Kenneth Rosen 2,3,4 What is computability? • `Computation’ is a pre-theoretical notion, The Collins Dictionary and Thesaurus (1987) reflects an outdated view: – “computation: a calculation involving numbers or quantities.” – “calculate: to solve by a mathematical procedure” • computing scientists have turned it into a technical term Informal explication of `computation’ • Suppose you want to achieve some goal • Suppose you have a procedure for doing this (stated in some objective way) • Procedure relies on primitives that are unproblematic What makes … • a good cooking recipe? • good instructions for using data projector? Desirable: (e.g., think of long division) • Statement of procedure is explicit, clear and unambiguous; no inventiveness required • Statement is finite, and requires finite resources (e.g., money, memory) • The outcome of the procedure can be trusted This is called an effective procedure, a computation, or an algorithm. History • Gottlob Leibnitz (1700) “Calculemus”: let’s decide all disputes by computation • David Hilbert (1900): “Can all questions of mathematics be answered algorithmically?” • Kurt Goedel (1931): This is not possible, even for number theory. (Result was independent of the notion of computation.) • Alan Turing (1936): The first formal model of computation, the Turing Machine Other formalisations: – Lambda calculus, – Recursive functions Church’s thesis Proven equivalent: – Turing machines – Lambda calculus – Recursive functions – Any general-purpose programming language Thesis: this is all there is to computation Some variations are still possible How about another requirement: “Goal is always achieved in finite time” (?) One answer: termination is always necessary This position has the advantage of clarity. Let us look at an example. Here is a simple program for determining whether a number n is prime: (in quasi-PASCAL, as in lecture notes) Problem: is n a prime number? program prime (input, output); var n, i : integer; begin read (n); i := 2; while (i<n) and (n mod i <> 0) {n is not divisible by i} do i := i+1; if i=n then writeln(“n is prime”) else writeln(“n is not prime”); end • This procedure is slow, but we will not worry about speed for now (until week 22: complexity) • Program terminates. If it did not, it would not be useful. Note: it addresses a (decision) problem: a decision YES/NO • What if the goal of the program is to find something? What if that something may not exist? • Example: find the smallest prime number that contains a string of n sevens. “Find prime containing string of n sevens” • One approach: Set n, then run previous procedure with increasingly large numbers. For each prime, test whether it contains a string of n sevens, until you have found one. • For some n, you may never find one. Does this make the procedure noneffective? Is it not a computation? Some people have answered NO: (second response) “Decision problems require termination. Other procedures do not.” Both responses are legitimate: The meaning of computation (effective procedure, algorithm) is not cast in stone Another example (Harel) • Algorithm A: (1) while x<>1 do x:=x-2; (2) stop • Clearly, A terminates iff x is odd Another example (Harel) • Algorithm B: (1) while x<>1 do if x is even then do x:=x/2; else do x:=3x+1 (2) stop • Try it out with inputs x=3, x=7 (1) while x<>1 do if x is even then do x:=x/2; else do x:=3x+1 (2) stop • x=3 trace: x:=10,5,16,8,4,2,1 • x=7 trace: x:=22,11,34,17,52,26,13,40,20,10,5,16,8,4,2,1 • B terminates for any known input, but no proof was available as to whether B must always terminate. • The strict view implies that we don’t know whether B is an algorithm. (We won’t take a stand here) Questions about algorithms • Does the algorithm terminate? [More about this below] • How long does it take? [Weeks 22,23] • How much memory does it use? • Does the algorithm always achieve the intended goal? [Correctness: other courses] Proof methods • Enumeration • Diagonalisation • Reduction Enumeration Structure of proof: there exist n algorithms there exist m problems m>n --------------------------------- therefore some problems don’t have an algorithm • Complication: the relevant numbers are large • Is it meaningful to ask how many programs there are? - YES • Georg Cantor (1845-1918) invented the calculus of infinite numbers. • Initially controversial, now very influential • Cantor’s starting point: N = the set of natural numbers = {0,1,2,…}. How many elements? Not 0, not 1, not 2, … • Cantor called this number 0 (the set is countably infinite). • The next number is called 1 the next one 2 and so on. (Together: uncountably infinite) • Question: what is 1 ? Cantor’s key principle: “Sets A and B have the same cardinality iff there exists a 1-1 correspondence between them”. (Also called bijection) Let A=N. Then how about 1. B={1,2,3,4,…}? 2. B={1,3,5,7,…}? 1. B={1,2,3,4,…}? Yes, use f:AB such that f(x)=x+1 A: 0 1 2 3 4 5 B: 1 2 3 4 5 6 f is a function, since xA yB(f(x,y)) f(x,y) & f(x,z) implies y=z f is an injection, since f(x,y) & f(z,y) implies x=z f is a surjection, since yB xA(f(x,y)) 2. B={1,3,5,7,…}? Yes, B has the same cardinality as N. To see this, use f:AB such that f(x)=2x+1 A: 0 B: 1 1 3 2 5 3 7 4 9 5 11 Note that a 1-1 correspondence with N is an enumeration. • So, many sets are equally large as N. They all have cardinality 0 • One could easily think that all infinite sets have this cardinality: there would not exist a cardinality 1 • This turns out to be false. In fact, the real numbers are a larger set than N. – (In fact, for all sets A, the power set of A is larger than A) Key observation: |||| > ||N|| Important for two reasons: 1. We need this fact for our proof by enumeration 2. Proof of |||| > ||N|| is seminal Proof proceeds by Reductio Ad Absurdum: By deducing a contradiction from p, we prove that ¬p Here goes: • Suppose was countably infinite (i.e., its cardinality is 0) • Then we could completely enumerate its elements: a1. (first real number) a2. (second real number) a3. (etc.) • In decimal notation, real numbers have 0 digits: a1 = <a11 , a12 , a13 ….> a2 = <a21 , a22 , a23 ….> a3 = <a31 , a32 , a33 ….> Now construct a number n(ew)=def <a11 +1 ,a22 +1 ,a33 +1, …> n differs from the i-th number on the i-th digit Which of the enumerated numbers is n? • n is not a1 because of a11 • n is not a2 because of a22 Etc. So, n is not in the list! Consequently, the enumeration was not complete after all • We have reached a contradiction: the enumeration is complete and incomplete • Adding the number n does not help … • (Homework: does the argument apply to the set Q of all fractions?) • Diagonalisation is a very productive proof method in computability Now back to the first proof method: Enumeration • Enumeration is much simpler than diagonalisation. Two questions to ask: (1) How many computations are there? (2) How many things are there to compute? • Regarding (1): Let’s focus on programs • Regarding (2): Let’s focus on numerical functions: from integers to integers (called integer functions) Enumeration (1) • Regarding simplification (1): “computations ~ programs” • This is a nontrivial step. In computability, people are usually more careful: they work with a formalism that is as simple as possible, while still allowing every reasonable `computation’: e.g., Turing machines. • Church’s thesis is not an exact result, yet it is practically universally believed • Relevance for us: if we prove something about what can be computed using PASCAL or JAVA, then we assume it to hold for any programming language that anyone could think of • Hence the simplification “computations ~ programs” Enumeration (2) Regarding simplification (2): things to calculate ~integer functions This is unproblematic. (If some integer functions are uncomputable then some problems are uncomputable) Noncomputability proof by enumeration: (1) The number of programs is 0. (2) The number of integer functions is 1 (3) Therefore some integer functions cannot be programmed (4) Therefore some integer functions are uncomputable [by Church’s thesis] Noncomputability proof by enumeration: (1) The number of programs is 0. Proof: - The number of programs whose length is 1 character can be enumerated - The number of programs whose length is 2 characters can be enumerated. Etc. - Arrange these enumerations head to tail, and the result is another enumeration Noncomputability proof by enumeration: (2) The number of integer functions is 1 Proof: by diagonalisation: Suppose there existed an enumeration of integer functions: F1, F2, F3, … . Then construct a function N(ew) using the following specification: • N differs from F1 in its value for arg. 1 • N differs from F2 in its value for arg. 2 • Etc. More concretely, we might define • N(1)=F1(1)+1 • N(2)=F2(2)+1 • N(3)=F3(3)+1 • Etc. • It follows that not all integer functions are computable • (Note that the argument can be applied to non-integer functions if we number their arguments and values.) Back to Cantor • Cantor proved that |||| > ||N|| • The proof does not show that nothing may lie in between. The Continuum Problem: – “Is any set larger than N but smaller than ?” – Equivalent: is |||| = 1 ? • One of the great open questions of pure mathematics. Nothing computational appears to hinge on it • This is not true for the other open problem that we shall meet (week 23)