Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
::ICS 804:: Theory of Computation - Ibrahim Otieno [email protected] +254-0722-429297 SCI/ICT Building Rm. G15 ICS 804 ICS 804 - Delivery ±60 contact hours 4.5 hours a week Tuesday 7:00 8:30 Thursday 5:30 8:30 Assessment CAT Assignment Exams Theoretical course, but Interactivity Exercises (reflect exam questions) Laboratory sessions ICS 804 - Delivery Online access to course materials: ◦ Notes ◦ Slides ◦ Assignments http://learning.uonbi.ac.ke Login: ◦ Username: registration number (P58/10202/2010) ◦ Password: msc ICS 804 – Course Text Introduction Computation from the point of view of mathematics (discrete mathematics) Introduction to computability and complexity theory What kind of problems are inherently computable and which options are there? Regular languages and finite-state automata Very recent field started in the 1930s Course Outline Mathematical Preliminaries Turing Machines Recursion Theory Markov Algorithms Register Machines Regular Languages and finite-state automata Aspects of Computability Course Outline Mathematical Preliminaries Turing Machines Recursion Theory Markov Algorithms Register Machines Regular Languages and finite-state automata Aspects of Computability Discrete Mathematics Theory of computability = new branch of mathematics Presupposes background in discrete mathematics Topics: ◦ Set-theoretic concepts ◦ Formal language theory ◦ Functions ◦ Big-O notation ◦ Propositional logic ◦ Proof Techniques ◦ Number-theoretic predicates Sets Sets Elements of a set: may be physical objects Set: never physical – is an abstract concept a S – a member of set S Members of sets may be sets S = {0, 1, {0, 1}} Cardinality: card(S) = 3 Singleton: S = {a} – set containing single element Sets lists (unordered, elements are unique) Set = family = class Specifying (Naming) Sets Finite sets {a,e,i,o,u} – elements can be listed Infinite sets: come from mathematics ◦ enumerate elements in an extended sense ◦ ◦ Natural numbers N = { 0,1,2,…} Integers Z = {…,-2,-1,0,1,2,…} Set abstraction ◦ ◦ ◦ {n|n is prime} or {n:n is prime} {n|…n…} or {n:…n…}: abstraction operator: the set of all n such that n… {n|n N & 2|n}: all even natural numbers Specifying (Naming) Sets … By convention: ◦ Use variables n,m,k,j,…: natural numbers / integers ◦ Use variables x,y,z,…: rational/real numbers ◦ Set variables: uppercase letters (A,B,C, …S) ◦ Universal set: U – all elements in current domain U by set abstraction {x|x=x} ◦ Empty Set: - with no element by set abstraction {x|xx} Subsets A = {0,1,2,3,4} A is subset of N or AN A is a subset of B ◦ If for any x, x A implies x B Consequences: ◦ ◦ Every set is a subset of itself i.e. A A, for any set A A, for any set A “proper subset”: non-identical subset {1,2,3} {1,2,3,4} Power set – all subsets of a set ◦ ◦ ◦ For a set S: P(S) = { A|A S } What is the power set of S = {1,2,3} ? Card(P(S) = 2card(s) for a finite set S Tuples A tuple is an ordered list of elements Ordered pair: <a,b> = <c,d> iff a = c and b = d defined Ordered triple: <a,b,c> 0-tuple: A = {<>} 1-tuple: A = <a> For any two sets A and B: ◦ ◦ ◦ ◦ ◦ ◦ are said to be disjoint if AB = Card(AB) Card(A) + Card(B) Card(AB) = Card(A) + Card(B) iff AB = Compliment of B relative to A denoted A\B is defined as {x| x A and x B} We write BC and speak of compliment of B for the set U\B Symmetric difference denoted by AB is as {x| (x A and x B) or (x B and x A) } Some set properties A∪U=U A∩= C=U UC = Idempotent Law: For any set A, ◦ A ∪A =A ◦ A ∩A =A Identity Law: For any set A, ◦ A ∪ =A ◦ A ∩ U =A Some set properties Commutative Law: For any two sets A and B ◦ A ∪ B = B ∪A ◦ A ∩ B = B ∩A Associative Law: For any three sets A, B and C ◦ (A ∪ B) ∪ C = A ∪ (B ∪ C) ◦ A ∩ (B ∩ C) = (A ∩ B) ∩ C Distributive Law: For any three sets A, B and C ◦ A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C) ◦ A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) De Morgan’s Law: For any two sets A and B ◦ (A ∪ B)' = A' ∩ B' ◦ (A ∩ B)' = A' ∪ B' Special Sets N: set of natural numbers closed under addition and multiplication not closed under subtraction nor division Z: set of integers closed under addition, subtraction and multiplication Q: set of rational numbers Q = {x|x = p/q for some p,q Z with q 0} closed under addition, subtraction and multiplication Q\{0} is closed under division R: set of real numbers closed under addition, subtraction and multiplication R\{0} closed under division We define a partition of a set A to be a family of mutually disjoint subsets such that A = A1 ∪ A2 ….An Exercises Exercise 1 If A and B are two sets, then A ∩ (A ∪ B) equals ◦ (A) A (B) (C) (D) none of these Exercise 2 The set (A ∪ B ∪ C) ∩ (A ∩ B' ∩ C')' ∩ C' is equal to ◦ (A) B ∩ C' (B) A ∩ C (C) B' ∩C' (D) None of these Formal Language Theory Formal Language Formal language: specific kind of set Formal = not natural language, programming language, … but: NLP try to formalize natural language Ignoring semantics: ◦ possible to regard natural languages like Spanish and programming languages such as PROLOG as formal Formal Language Alphabet : finite set of symbols, e.g. = {a,b} Word over alphabet : finite string of elements of allowing repetition Length of word w = |w| = number of symbol occurrences in w Card() = m then there exist mn distinct words over of length n ◦ For alphabet = {a,b} we have 2n distinct words over of length n Formal Language Special superscript notation: anbm word consisting of n times a and m times b Example: a3b4a2 aaabbbbaa Distinct occurrences: na(aba)=2 nb(aba)=1 Formal Language Null word: word of length 0 ◦ ◦ ◦ Zero occurrences of a and b = a0 na() = 0 and nb() = 0 Universal Language: * set of all words over Language over : any subset of * ◦ ◦ ◦ Empty language: - containing no word Universal language: * - containing all words Unit language L = {w} – containing word, w, only Alphabet = {a,b} ◦ {w *: |w| = 3} = aaa, aab, aba, abb, baa, bab, bba, bbb} Operations on words Concatenation: ww’ of words w and w’ w is aba and w’ is bba ww’ = ababba Reverse of a word: wR abbR = bba Palindromes over is the language {w*|where w=wR} Palindrome Examples Examples: Radar – level – mum – gig “Madam, in Eden, I’m Adam” “A man, a plan, a canal: Panama” “Doc, note. I dissent. A fast never prevents a fatness. I diet on cod.” (Peter Hilton) Longest 1-word English palindrome: detartrated - (to remove tartrates) Longest 1-word palindrome in the world (Finnish) saippuakivikauppias - (lye dealer) Swahili palindromes: amesema, ataufuata, akatutaka Longest palindrome: 31358 words Operations on words A word w’ is a prefix of a word w if w can be written w’w’’ for some word w’’ ◦ ,a,ab,abb,abba are prefixes of the word abba (first 4 are proper) A word w’’ is a suffix of a word w if w can be written w’w’’ for some word w’ ◦ ,a,ba,bba,abba are suffixes of the word abba (first 4 are proper) Operations on words A word w’’ is a substring of word w if w can be written as w’w’’w’’’ for some words w’ and w’’ abba: a ab bb ba abb bba abba Exercises 1 = {a,b,c} 2 = {a,b,ca} 3 = {a,b,Ab} Determine to which * each word belongs aba bAb cba cab caab baAb Solution 1 = {a,b,c} 2 = {a,b,ca} 3 = {a,b,Ab} Determine to which * each word belongs aba 1,2,3 bAb 3 cba 1 cab 1,2 caab1,2 baAb 3 Exercises Calculate how many words there are in the following languages. List 3 elements in each of them. Which language contains ◦ ◦ ◦ ◦ *, where = {a,b,c} {w*| |w| 3}, where = {a,b} {w*| |w| =4}, where = {a,b} {anb|n is prime} Solution *, where = {a,b,c} {w*| |w| 3}, where = {a,b} 15 words (23+22+21+20) = {, a, b, ab, ba, bb, aa, aaa, baa, aba, aab, bba, bab, abb, bbb} {w*| |w| =4}, where = {a,b} Infinite number of words {, abba, ccccba, aaccccccccc, …} 16 words (24) = {aaaa, aaab, aaba, abaa, baaa, aabb, abba, bbaa, baba, abab, baab, abbb, babb, bbab, bbba, bbbb} {anb|n is prime} Infinite number of words {aab, aaab, aaaaab, …} Exercises Let = {a,b,c,d}. List as many words as you can that are words in the language {w *| |w| = 3 and w is a word in the English language} Solution Let = {a,b,c,d}. List as many words as you can that are words in the language {w *| |w| = 3 and w is a word in the English language} dad cab bad dab add cad baa Exercises Suppose w is ab and w’ is bab. Identify each of the following words: ww’ wR (ww’)R ww’R Solution Suppose w is ab and w’ is bab. Identify each of the following words: ww’ abbab wR ba (ww’)R babba ww’R abbab Exercise Which of the following languages are palindrome languages ◦ ◦ ◦ ◦ ◦ wwR for any w * wR for any palindrome w * ww for any palindrome w * wawR for any w * ww’wR for any w * and any palindrome w’ * ◦ Solution Which of the following languages are palindrome languages ◦ yes ◦ yes ◦ yes ◦ yes ◦ yes ◦ yes wwR for any w * wR for any palindrome w * ww for any palindrome w * wawR for any w * ww’wR for any w * and any palindrome w’ * Mapping and Functions Mapping and Functions Mapping: association of members of one set with members of another (not necessarily different) set First: domain Second: codomain or range Members of domain serve as: arguments Members of codomain serve as: values Mapping f between Dom(f)=A and Cod(f)=B f: A B A mapping f is a function if each member of Dom(f) is mapped to one and only one member of Cod(f) Function Function = single valued mapping Complete description of a mapping f consists of: a) specification of Dom(f) and Cod(f) b) description of values of f for any member of Dom(f) taken as arguments Examples f: N N f(n) = n + 1 Unary constant-7 function – returns 7 for all n C17: N N C17(n) = 7 f: * * f(w) = wR Functions Examples …continued Floor function f: R Z f(x) = [x] = greatest integer ≤ x Ceiling function f: R Z f(x) = [x] = least integer ≥ x Multi-valued mapping ◦ f: Z Z f(n) = n ± 3 f(n) = n 3 Not a function Functions Image of a function f defined as Image(f) is subset of Cod(f) consisting precisely of the values of f for members of Dom(f) Precisely in set language: Image(f) = {yCod(f)|y=f(x) for some x Dom(f)} Number-theoretic Functions Number-theoretic functions: map natural numbers to natural numbers: f: N N Unary number-theoretic functions f(n) = n + 1 Binary number-theoretic functions f: N 2 N f(n,m) = n + m K-ary number-theoretic functions f: N k N 0-ary number theoretic functions C02: N 0 N C02() = 2 Injective (1-to-1) Functions A function f is 1-to-1 or injective if no two distinct elements of Dom(f) are mapped to one and the same member of the Cod(f) Examples: Successor function f: N N f(n) = n + 1 Word reversal function f: * * f(w) = wR Ceiling, floor and constant-7 functions not injective Surjective (onto) Functions A function f is onto or surjective provided that every element of Cod(f) is a value of f for at least one element of the Dom(f) Cod(f) = Image (f) Examples: Word reversal function f: * * f(w) = wR Ceiling function f: R Z f(x) = [x] = least integer ≥ x Successor function not surjective – 0 not successor A function that is both injective and surjective is termed as 1-1 correspondence or a bijective function Functions forming operations Inverse For two functions g and f: g: N N g(n) = n/2 f: N N f(n) = 2n g is the inverse of f g = f-1 The inverse operation (though not always defined) -1is; Unary function forming operation Composition For two functions g and f: g: * N where ={a,b} g(w) = na(w) f: N N f(n) = 3n + 2 h: * N h(w) = f(g(w)) h: composition of g and f (h = g f) The composition operation is: Binary function forming operation Partial Functions For a function g: g: N 2 N g(n,m) = n – m The function g is undefined for pairs <n,m> where n < m Number-theoretic functions that are undefined for some n N are Partial functions. Partial Functions Let f(n) and g(n) be partial number-theoretic functions We can define a new partial number-theoretic function h(n) = max{ f(n), g(n) } This means that h(n) is defined just in case both f(n) and g(n) are defined for any n Dom(h) = Dom(f) Dom(g), we have h(n) is the maximum of f(n) and g(n) Polynomial Functions We restrict ourselves to unary polynomial functions Polynomial in n of degree k is any expression of form: aknk + ak-1nk-1 + ak-2nk-2 + ….. + a1n1 + a0n0 where k is any natural constant and coefficients ak , ak-1 , ak-2 , ….., a1 + a0 are integer constants with ak ≠ 0 Polynomials of degree 0 are numerals Polynomials in n are not themselves natural numbers or integers; they are expressions Polynomial Functions For example: 2n5 + 2n3 + 4n2 +16n + 7 is a polynomial of degree 5 The unary polynomial function is defined by: f: N N f(n) = 2n5 + 2n3 + 4n2 +16n + 7 f(2) = ? Polynomial functions with negative coefficients may be partial Sometimes we write p(n) for a unary polynomial function of n For p1(n) and p2(n) => p2[p1(n)]) – result of replacing every occurrence of n in p2(n) with p1(n) Big-O Notation Big-O Big-O (micron) notation provides standard means of comparing growth of number-theoretic functions Analyze time and space requirements of algorithms Typical calculation: growth of a function as argument n goes to infinity Monotone increasing: for all n1, n2 N, we have n1< n2 implies f(n1) f(n2) Strictly monotone increasing: f(n1)<f(n2) Big-O Four partial number-theoretic functions: k(n) = 2n h(n) = n2 + 1 g(n) = 3n f(n) = 7[log2 n] Big-O Big-O Which are (strictly) monotone increasing ? Big-O Four partial number-theoretic functions k(n) = 2n h(n) = n2 + 1 g(n) = 3n f(n) = 7[log2 n] Value 2: k(2) = 4 h(2)=5 g(2) = 6 Value 5: k(5) = 32 h(5)=26 g(2) = 15 f(2)=7 f(2)=14 Big-O Fastest growing function? Slowest? Big-O For small numbers f is higher, but it is k that grows the fastest for large numbers. Big-O expresses how fast a function grows relative to another Function f(n) is O(g(n)) ◦ g(n) is at least as great as f(n); ultimately for all sufficiently large n Similarly; g(n) is O(h(n)) and h(n) is O(k(n)) Transitivity holds: ◦ If f(n) is O(g(n)) and g(n) is O(h(n)) then f(n) is O(h(n)) Big-O Definition: Let f(n) and g(n) be partial number-theoretic functions. Then function f(n) is O(g(n)) if there exist natural number constants C and n0 such that for any nn0 we have f(n) C.g(n) whenever both f(n) and g(n) are defined. e.g. Function f(n) is O(g(n)): with C=1 and n0=5 4 5=n0 7 16 f(n) = 7[log2 n] 14 14 14 28 C.g(n) = 1.3n 12 15 21 48 Big-O • • • • • • • f1 is O((f2)) f3 is O((f4)) f5 is O((f6)) ….. ….. ….. f9 is O((f10)) • Growth rate of a function ftime is O(f72(n)) for instance In other words function ftime is O(n2) • Big-O Big-O Function f(0) is O(0) Function f(n) that is O(1) iff f(n) is bounded above by a constant for sufficiently large arguments n Function f(n) that is O(n) has linear growth Function f(n) is O(n2) or O(n3): polynomial growth Function f(n) is O(2n) or O(3n): exponential growth Exercises (a) (b) Characterize these functions as ◦ monotone increasing ◦ strictly monotone increasing ◦ neither (c) (e) (d) (f) Solution (a) (c) (e) (b) (d) (f) • • • • • • Strictly monotone Neither Strictly monotone Monotone Neither Monotone Exercises Which of the following functions are O(1) ◦ ◦ ◦ ◦ ◦ f(n) = [log2 n] g(n) = [log10 n] h(n) = min(n,100) j(n) = 4n2 + 7n + 3 k(n) = 5n5 + n3 + n + 68 Solution Which of the following functions are O(1) ◦ f(n) = [log2 n] ◦ g(n) = [log10 n] ◦ h(n) = min(n,100) only this one is O(1) ◦ j(n) = 4n2 + 7n + 3 ◦ k(n) = 5n5 + n3 + n + 68 Propositional Logic Proposition Proposition: any statement with a truth value Any statement that is either true or false Examples: ◦ For all n in P, n3 – n is divisible by 6 ◦ N2 is O(n3) ◦ 6 is a prime number ◦ Nairobi is located in Kenya ◦ Goldbach’s Conjecture: every even integer strictly greater than 4 is the sum of two primes ◦ Don’t forget to vote ◦ Is Russian beer the best beer in the world? ◦ We define the degree of a vertex v within an undirected graph G to be the number of edges in G incident upon v. Notation PL: infinite symbol set of ◦ lower-case letters of the Roman alphabet, p,q,r,p’,q’,r’,p’’,q’’,r’’,… which stand for propositions: sentence letters ◦ Propositional connectives: & ◦ Aggregation signs - (), [], {} Language of propositional logic- set of strings over alphabet PL Negation Symbol: p p T F F T p: John is tall p: John is not tall Unary connective Conjunction Binary connective: symbol - & p q p&q T T T F T F T F F F F F For conjuncts p and q p: John is tall q: Andrew is short p&q: John is tall and Andrew is short Disjunction Binary connective: symbol - p q pq T T T F T T T F T F F F Inclusive or For disjuncts p and q p: John is tall q: Andrew is short p q: Either John is tall or Andrew is short Conditional Used to express conditional statements Binary connective: symbol p q pq T T T F T T T F F F F T p: It is raining q: The streets are wet Antecedent p and consequent q False wherever antecedent is true and consequent is false Conditional Binary connective: Not quite the same semantics as in natural language If Kenya is in Europe, then 2+2=4 = false Biconditional Binary connective: symbol p q pq T T T F T F T F F F F T If and only if p: 2+2 = 4 q: 1+1 = 2 True if component sentence letters have same truth values Semantic Issues Sentences of propositional logic classified as: Tautologous ◦ True under all assignments of component sentence letters ◦ pp Contradictory ◦ False under all assignments of component sentence letters ◦ p&p Contingent ◦ Sentences that are neither tautologous or contradictory ◦ p & q and p q Semantic Issues Satisfiable ◦ A sentence is satisfiable if for at least one assignment of the component letters the sentence holds true ◦ Not contradictory ◦ Truth tables: decision procedure for satisfiability ◦ The satisfiability problem for propositional logic is decidable Equivalence Two sentences S1 and s2 are (logically) equivalent if for any given assignment of truth values to component sentence letters, the resulting truth value of S1 is identical with value of S2 Idempotence, Associativity, Commutativity, Distributivity and DeMorgans Laws also hold PROOF TECHNIQUES Proof Techniques Conditional Proof pq assume truth of the antecedent reason to the truth of consequent q Example 1 If I drink too much alcohol, I can’t drive a car • When I drink alcohol, the alcohol gets absorbed in my blood. My blood reaches my brain, where it will distort its functionality, reducing my coordination abilities. To drive a car I need coordination. Proof Techniques Conditional Proof Example 2 If Jones is elected, then the treasury will be bankrupted • Given that Jones is lavish and a spendthrift • He will spend vast amounts on annual banquets • Given his slackness in supervising his subordinates, there will be widespread corruption Therefore, if elected then the treasury will be bankrupted Proof Techniques Indirect Proof pq Proof by contradiction • 11 is not a prime number if it is not a prime number, it must have a non-trivial factor. But it is not 2,3,4,5,6,7,8,9,10 and it can’t be larger than n. This is a contradiction, so 11 is a prime after all • • • • Appropriate when p is unstructured. Derive a contradiction of the form p & p. Having assumed p is true, continue to show that p & p is true. Conclude then that p is not true, thereby showing p is true. Number-theoretic Predicates Number-theoretic Predicates • Properties of objects, relations, holding between pairs of objects, relations, holding among triples of objects are expressed using predicates. •If the objects are members of N, then one speaks of number-theoretic predicates •e.g. prime(n) is a unary number-theoretic predicate that is satisfied by 2 and 3, but not by 4 or 6 •e.g. predicate n|m hold just in case m is a multiple of n so that 2|6 and 2|0 hold Number-theoretic Predicates • The extension of a k-ary number-theoretic predicate is the set of all and only those k-tuples of natural numbers satisfying that predicate extension of the predicate prime(n) is the set {n N|prime(n)} extension of cube_of(n,m) is the infinite set of ordered pairs {<0,0>,<1,1>,<8,2>,<27,3>,<64,4>,…} Number-theoretic Predicates • Predicates can be combined using the five propositional connectives: & C1(n1,n2,…nk) & C2(n1,n2,…nk) is true if k-tuple <n1,n2,…,nk> satisfies both C1(n1,n2,…nk) and C2(n1,n2,…nk) • • A binary predicate C(n,n) is said to be reflexive if, for any and all n N, we have C(n,n) A binary predicate C(n1,n2) is said to be symmetric if, for any n, m N; C(n,m) implies C(m,n) Notation Quantifier: may play a role in specification of predicates ◦ Universal quantifier (n)(C1(n) C1(n)) C1(n): prime(n) Given any natural number n, either n is prime or it is not Given any natural number n is prime and n> 2 is odd ? ◦ Existential quantifier (n)(C1(n) & C2(n)) C1(n): prime(n) C2(n): odd(n) There exists an n such that n is prime and n is not odd There is no prime greater than 2 that is even? Exercise Given: the following predicates C1(n) C2(n,m,k) C3(n,m,k) with n,m,k N C1(n1) n1 is prime C2(n1,n2,n3) n1 = n2 + n3 C3(n1,n2,n3) n1 = n2 . n3 For each of the predicates below, determine whether it is satisfied by the pair <4,12> n=4 m=12 (k)C2(n,m,k) (k)[C2(k,n,m) & C1(k)] (k)C3(n,m,k) (k)C3(k,n,m) (k)[C3(k,n,m)&C1(k)] (kn)[C3(m,k,n) & C1(k)] Exercise Given: the following predicates C1(n) C2(n,m,k) C3(n,m,k) with n,m,k N C1(n1) n1 is prime C2(n1,n2,n3) n1 = n2 + n3 C3(n1,n2,n3) n1 = n2 . n3 For each of the predicates below, determine whether it is satisfied by the pair <4,12> n=4 m=12 (k)C2(n,m,k) no (k)[C2(k,n,m) & C1(k)] no (k)C3(n,m,k) yes (k)C3(k,n,m) yes (k)[C3(k,n,m)&C1(k)] no (kn)[C3(m,k,n) & C1(k)] yes