Download Lecture 5

Lecture 5
• Topics
– Proof of the existence of unsolvable problems
• Proof Technique
– There are more problems/languages than there are
programs/algorithms
– Countable and uncountable infinities
1
Overview
• We will show that there are more problems
than programs
– Actually more problems than programs in any
computational model (programming language)
• Implication
– Some problems are not solvable
2
Preliminaries
Define set of problems
Observation about programs
3
Define set of problems
• We will restrict the set of problems to be the
set of language recognition problems over
the alphabet {a}.
• That is
– Universe: {a}*
– Yes Inputs: Some language L subset of {a}*
– No Inputs: {a}* - L
4
Set of Problems
• The number of distinct problems is given by
the number of languages L subset of {a}*
– 2{a}* is our shorthand for this set of subset
languages
• Examples of languages L subset of {a}*
–
–
–
–
0 elements: { }
1 element: {/\}, {a}, {aa}, {aaa}, {aaaa}, …
2 elements: {/\, a}, {/\, aa}, {a, aa}, …
Infinite # of elements: {an | n is even}, {an | n is
prime}, {an | n is a perfect square}
5
Infinity and {a}*
• All strings in {a}* have finite length
• The number of strings in {a}* is infinite
• The number of languages L in 2{a}* is
infinite
• The number of strings in a language L in
2{a}* may be finite or infinite
6
Define set of programs
• The set of programs we will consider are
the set of legal C++ programs as defined in
earlier lectures
• Key Observation
– Each C++ program can be thought of as a finite
length string over alphabet SP
• SP = {a, …, z, A, …, Z, 0, …, 9, white space,
punctuation}
7
Example
int main(int A[], int n)
int i, max;
{27 characters including newline}
{12 characters including initial tab}
{1 character: newline}
if (n < 1)
return (“Illegal Input”);
max = A[0];
for (i = 1; i < n; i++)
{12 characters}
{28 characters including 2 tabs}
{13 characters}
{…}
if (A[i] > max)
max = A[i];
return (max);
}
8
Number of programs
• The set of legal C++ programs is clearly
infinite
• It is also no more than |SP*|
– SP = {a, …, z, A, …, Z, 0, …, 9, white space,
punctuation}
9
Goal
• Show that the number of languages L in
2{a}* is greater than the number of strings in
|SP*|
– SP = {a, …, z, A, …, Z, 0, …, 9, white space,
punctuation}
• Problem
– Both are infinite
10
How do we compare the relative
sizes of infinite sets?
Bijection (yes)
Proper subset (no)
11
Bijections
• Two sets have EQUAL size if there exists a
bijection between them
– bijection is a 1-1 and onto function between
two sets
• Examples
– Set {1, 2, 3} and Set {A, B, C}
– Positive even numbers and positive integers
12
Bijection Example
• Positive Integers Positive Even Integers
1
2
3
...
i
…
2
4
6
...
2i
...
13
Proper subset
• Finite sets
– S1 proper subset of S2 implies S2 is strictly
bigger than S1
• Example
– women proper subset of people
– number of women less than number of people
• Infinite sets
– Counterexample
• even numbers and integers
14
Two sizes of infinity
Countable
Uncountable
15
Countably infinite set S
• Definition 1
– S is equal in size (bijection) to N
• N is the set of natural numbers {1, 2, 3, …}
• Definition 2 (Key property)
– There exists a way to list all the elements of set
S (enumerate S) such that the following is true
• Every element appears at a finite position in the
infinite list
16
Uncountable infinity
• Any set which is not countably infinite
• Examples
– Set of real numbers
– 2{a}*, the set of all languages L which are a
subset of {a}*
• Further gradations within this set, but we
ignore them
17
Proof
18
(1) The set of all legal C++
programs is countably infinite
• Every C++ program is a finite string
• Thus, the set of all legal C++ programs is a
language LC
• This language LC is a subset of SP*
19
For any alphabet S, S* is
countably infinite
• Enumeration ordering
– All length 0 strings
• |S|0 = 1 string: l
– All length 1 strings
• |S| strings
– All length 2 strings
• |S|2 strings
–…
• Thus, SP* is countably infinite
20
Example with alphabet {a,b}
• Length 0 strings
– 0 and l
• Length 1 strings
– 1 and a, 2 and b
• Length 2 strings
– 3 and aa, 4 and ab, 5 and ba, 6 and bb, ...
• Question
– write a program that takes a number as input and
computes the corresponding string as output
21
(2) The set of languages in 2{a}*
is uncountably infinite
• Diagonalization proof technique
– “Algorithmic” proof
– Typically presented as a proof by contradiction
22
Algorithm Overview
• To prove this set is uncountably infinite, we
construct an algorithm D that behaves as
follows:
– Input
• A countably infinite list of languages L[] subset of {a}*
– Output
• A language D(L[]) which is a subset of {a}* that is not
on list L[]
– Declaration of algorithm D
language D(language L[ ])
23
Visualizing D
List L[]
L[0]
L[1]
L[2]
Algorithm D
Language
D(L[]) not in
list L[]
L[3]
...
24
Why existence of D implies result
• If the number of languages in 2{a}* is countably
infinite, there exists a list L[] s.t.
– L[] is complete
• it contains every language in 2{a}*
– L[] is countably infinite
• The existence of algorithm D implies that no
list of languages in 2{a}* is both complete and
countably infinite
– Specifically, the existence of D shows that any
countably infinite list of languages is not complete
25
Visualizing L[ ]
L[0]
L[1]
L[2]
L[3]
L[4]
l
a
aa aaa aaaa ...
IN
IN
IN
IN
IN
IN OUT IN OUT IN
OUT OUT OUT OUT OUT
OUT IN
IN OUT OUT
•#Rows is countably infinite
•Given
•#Cols is countably infinite
• {a}* is countably infinite
OUT IN OUT OUT IN
...
• Consider each string to be a feature
– A set contains or does not contain each string
26
Constructing D(L[ ])
• We construct D(L[]) by using a unique feature
(string) to differentiate D(L[]) from L[i]
– Typically use ith string for language L[i]
– Thus the name diagonalization
D(L[]) l
a
L[0]
L[1]
L[2]
OUT
IN IN
L[3]
L[4]
OUT IN
aa aaa aaaa ...
IN
IN
IN
IN OUT
IN IN OUT IN
OUT OUT OUT
IN OUT OUT
IN OUT
IN OUT
OUT IN OUT OUT OUT
IN
...
27
Questions
L[0]
L[1]
L[2]
L[3]
L[4]
l
a
aa aaa aaaa ...
IN
IN
IN
IN
IN
IN OUT IN OUT IN
OUT OUT OUT OUT OUT
OUT IN
IN OUT OUT
OUT IN OUT OUT IN
...
• Do we need to use the diagonal?
– Every other column and every row?
– Every other row and every column?
• What properties are needed to construct D(L[])?
28
Visualization
All problems
Solvable Problems
The set of solvable problems is a proper subset of the set
of all problems.
29
Summary
• Equal size infinite sets: bijections
– Countable and uncountable infinities
• More languages than algorithms
– Number of algorithms countably infinite
– Number of languages uncountably infinite
– Diagonalization technique
• Construct D(L[]) using infinite set of features
• The set of solvable problems is a proper
subset of the set of all problems
30