Download Languages and Finite Automata

Mathematical Preliminaries
Courtesy Costas Busch - RPI
1
Mathematical Preliminaries
• Sets
• Functions
• Relations
• Graphs
• Proof Techniques
Courtesy Costas Busch - RPI
2
SETS
A set is a collection of elements
A  {1, 2, 3}
B  {train, bus, bicycle, airplane}
We write
1 A
ship  B
Courtesy Costas Busch - RPI
3
Set Representations
C = { a, b, c, d, e, f, g, h, i, j, k }
C = { a, b, …, k }
finite set
S = { 2, 4, 6, … }
infinite set
S = { j : j > 0, and j = 2k for some k>0 }
S = { j : j is nonnegative and even }
Courtesy Costas Busch - RPI
4
A = { 1, 2, 3, 4, 5 }
U
A
6
1
7
2
4
5
10
Universal Set:
8
3
9
all possible elements
U = { 1 , … , 10 }
Courtesy Costas Busch - RPI
5
Set Operations
A = { 1, 2, 3 }
B = { 2, 3, 4, 5}
B
A
• Union
A U B = { 1, 2, 3, 4, 5 }
2
1
3
4
5
• Intersection
U
A
B = { 2, 3 }
2
3
• Difference
A-B={1}
1
B - A = { 4, 5 }
Venn diagrams
Courtesy Costas Busch - RPI
6
• Complement
Universal set = {1, …, 7}
A = { 1, 2, 3 }
4
A = { 4, 5, 6, 7}
A
1
5
A
2
6
3
7
A=A
Courtesy Costas Busch - RPI
7
{ even integers } = { odd integers }
Integers
1
odd
2
3
even
0
5
6
4
Courtesy Costas Busch - RPI
7
8
DeMorgan’s Laws
U
A
U
AUB=A
B
B=AUB
Courtesy Costas Busch - RPI
9
Empty, Null Set:
={}
SU
=S
S
=
U
S-
= Universal Set
=S
-S=
Courtesy Costas Busch - RPI
10
Subset
A = { 1, 2, 3}
B = { 1, 2, 3, 4, 5 }
B
U
Proper Subset: A
U
A
B
B
A
Courtesy Costas Busch - RPI
11
Disjoint Sets
A = { 1, 2, 3 }
A
U
A
B = { 5, 6}
B=
B
Courtesy Costas Busch - RPI
12
Set Cardinality
• For finite sets
A = { 2, 5, 7 }
|A| = 3
(set size)
Courtesy Costas Busch - RPI
13
Powersets
A powerset is a set of sets
S = { a, b, c }
Powerset of S = the set of all the subsets of S
2S = {
, {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }
Observation: | 2S | = 2|S|
Courtesy Costas Busch - RPI
( 8 = 23 )
14
Cartesian Product
A = { 2, 4 }
B = { 2, 3, 5 }
A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }
|A X B| = |A| |B|
Generalizes to more than two sets
AXBX…XZ
Courtesy Costas Busch - RPI
15
FUNCTIONS
domain
4
A
1
2
5
If A = domain
range
B
f(1) = a
a
b
3
c
f : A -> B
then f is a total function
otherwise f is a partial function
Courtesy Costas Busch - RPI
16
RELATIONS
Let A & B be sets. A binary relation “R” from A to B
R = {(x1, y1), (x2, y2), (x3, y3), …}
Where xi  A and yi  B
R⊆AxB
xi R yi to denote (a, b)  R
e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1
Courtesy Costas Busch - RPI
17
Equivalence Relations
• Reflexive:
xRx
• Symmetric:
xRy
• Transitive:
x R y and y R z
yRx
xRz
Example: R = ‘=‘
•x=x
•x=y
• x = y and y = z
y=x
x=z
Courtesy Costas Busch - RPI
18
Equivalence Classes
For equivalence relation R
equivalence class of x = {y : x R y}
Example:
R = { (1, 1), (2, 2), (1, 2), (2, 1),
(3, 3), (4, 4), (3, 4), (4, 3) }
Equivalence class of 1 = {1, 2}
Equivalence class of 3 = {3, 4}
Courtesy Costas Busch - RPI
19
GRAPHS
A directed graph
e
node
b
d
a
c
• Nodes (Vertices)
V = { a, b, c, d, e }
• Edges
E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }
Courtesy Costas Busch - RPI
20
Labeled Graph
2
6
a
b
1
5
3
e
6
2
d
c
Courtesy Costas Busch - RPI
21
Walk
e
b
d
a
c
Walk is a sequence of adjacent edges
(e, d), (d, c), (c, a)
Courtesy Costas Busch - RPI
22
Path
e
b
d
a
c
Path is a walk where no edge is repeated
Simple path: no node is repeated
Courtesy Costas Busch - RPI
23
Cycle
base
a
3
2
e
b
1
d
c
Cycle: a walk from a node (base) to itself
Simple cycle: only the base node is repeated
Courtesy Costas Busch - RPI
24
Euler Tour
8
b
4
a
7
3
6
5
base
e
1
2
d
c
A cycle that contains each edge once
Courtesy Costas Busch - RPI
25
Hamiltonian Cycle
5
b
4
a
3
base
e
1
2
d
c
A simple cycle that contains all nodes
Courtesy Costas Busch - RPI
26
Finding All Simple Paths
e
b
d
a
c
origin
Courtesy Costas Busch - RPI
27
Step 1
e
b
d
a
c
(c, a)
origin
(c, e)
Courtesy Costas Busch - RPI
28
Step 2
e
b
d
a
(c, a)
(c, a), (a, b)
c
origin
(c, e)
(c, e), (e, b)
(c, e), (e, d)
Courtesy Costas Busch - RPI
29
Step 3
e
b
d
a
(c, a)
(c, a), (a, b)
c
origin
(c, a), (a, b), (b, e)
(c, e)
(c, e), (e, b)
(c, e), (e, d)
Courtesy Costas Busch - RPI
30
Step 4
e
b
(c, a)
d
a
c
(c, a), (a, b)
(c, a), (a, b), (b, e)
origin
(c, a), (a, b), (b, e), (e,d)
(c, e)
(c, e), (e, b)
(c, e), (e, d)
Courtesy Costas Busch - RPI
31
root
Trees
parent
leaf
child
Trees have no cycles
Courtesy Costas Busch - RPI
32
root
Level 0
Level 1
Height 3
leaf
Level 2
Level 3
Courtesy Costas Busch - RPI
33
Binary Trees
Courtesy Costas Busch - RPI
34
PROOF TECHNIQUES
• Proof by induction
• Proof by contradiction
Courtesy Costas Busch - RPI
35
Induction
We have statements P1, P2, P3, …
If we know
• for some b that P1, P2, …, Pb are true
• for any k >= b that
P1, P2, …, Pk imply Pk+1
Then
Every Pi is true
Courtesy Costas Busch - RPI
36
Proof by Induction
• Inductive basis
Find P1, P2, …, Pb which are true
• Inductive hypothesis
Let’s assume P1, P2, …, Pk are true,
for any k >= b
• Inductive step
Show that Pk+1 is true
Courtesy Costas Busch - RPI
37
Example
Theorem: A binary tree of height n
has at most 2n leaves.
Proof by induction:
let L(i) be the maximum number of
leaves of any subtree at height i
Courtesy Costas Busch - RPI
38
We want to show: L(i) <= 2i
• Inductive basis
L(0) = 1
(the root node)
• Inductive hypothesis
Let’s assume L(i) <= 2i for all i = 0, 1, …, k
• Induction step
we need to show that L(k + 1) <= 2k+1
Courtesy Costas Busch - RPI
39
Induction Step
height
k
k+1
From Inductive hypothesis: L(k) <= 2k
Courtesy Costas Busch - RPI
40
Induction Step
height
k
L(k) <= 2k
k+1
L(k+1) <= 2 * L(k) <= 2 * 2k = 2k+1
(we add at most two nodes for every leaf of level k)
Courtesy Costas Busch - RPI
41
Remark
Recursion is another thing
Example of recursive function:
f(n) = f(n-1) + f(n-2)
f(0) = 1, f(1) = 1
Courtesy Costas Busch - RPI
42
Proof by Contradiction
We want to prove that a statement P is true
• we assume that P is false
• then we arrive at an incorrect conclusion
• therefore, statement P must be true
Courtesy Costas Busch - RPI
43
Example
Theorem:
2
is not rational
Proof:
Assume by contradiction that it is rational
2
= n/m
n and m have no common factors
We will show that this is impossible
Courtesy Costas Busch - RPI
44
2
= n/m
Therefore,
2 m2 = 4k2
n2
2 m 2 = n2
is even
m2 = 2k2
n is even
n=2k
m is even
m=2p
Thus, m and n have common factor 2
Contradiction!
Courtesy Costas Busch - RPI
45
Languages
Courtesy Costas Busch - RPI
46
A language is a set of strings
String: A sequence of letters
Examples: “cat”, “dog”, “house”, …
Defined over an alphabet:
  a, b, c,, z
Courtesy Costas Busch - RPI
47
Alphabets and Strings
We will use small alphabets:
  a, b
Strings
a
ab
u  ab
abba
v  bbbaaa
baba
w  abba
aaabbbaabab
Courtesy Costas Busch - RPI
48
String Operations
w  a1a2 an
abba
v  b1b2 bm
bbbaaa
Concatenation
wv  a1a2 anb1b2 bm
Courtesy Costas Busch - RPI
abbabbbaaa
49
w  a1a2 an
ababaaabbb
Reverse
w  an a2a1
R
Courtesy Costas Busch - RPI
bbbaaababa
50
String Length
w  a1a2 an
Length:
w n
Examples:
abba  4
aa  2
a 1
Courtesy Costas Busch - RPI
51
Length of Concatenation
uv  u  v
Example:
u  aab, u  3
v  abaab, v  5
uv  aababaab  8
uv  u  v  3  5  8
Courtesy Costas Busch - RPI
52
Empty String
A string with no letters:
Observations:
, 
 0
w  w  w
abba  abba  abba
Courtesy Costas Busch - RPI
53
Substring
Substring of string:
a subsequence of consecutive characters
String
Substring
abbab
ab
abbab
abba
abbab
b
abbab
bbab
Courtesy Costas Busch - RPI
54
Prefix and Suffix
abbab
Prefixes
Suffixes

abbab
a
bbab
ab
bab
abb
ab
abba
b
abbab

Courtesy Costas Busch - RPI
w  uv
prefix
suffix
55
Another Operation
w  ww

w




n
n
Example:
Definition:
abba   abbaabba
2
w 
0
abba   
0
Courtesy Costas Busch - RPI
56
The * Operation
 * : the set of all possible strings from
alphabet 
  a, b
*   , a, b, aa, ab, ba, bb, aaa, aab,
Courtesy Costas Busch - RPI
57
The + Operation
 : the set of all possible strings from

alphabet  except 
  a, b
*   , a, b, aa, ab, ba, bb, aaa, aab,

   * 

  a, b, aa, ab, ba, bb, aaa, aab,
Courtesy Costas Busch - RPI
58
Languages
A language is any subset of
Example:
*
  a, b
*   , a, b, aa, ab, ba, bb, aaa,
Languages:

a, aa, aab
{ , abba, baba, aa, ab, aaaaaa}
Courtesy Costas Busch - RPI
59
Note that:
Sets
  { }  {}
Set size
{}    0
Set size
{}  1
String length
 0
Courtesy Costas Busch - RPI
60
Another Example
An infinite language
L  {a b : n  0}
n n

ab
L
abb  L
aabb
aaaaabbbbb
Courtesy Costas Busch - RPI
61
Operations on Languages
The usual set operations
a, ab, aaaa  bb, ab  {a, ab, bb, aaaa}
a, ab, aaaa  bb, ab  {ab}
a, ab, aaaa  bb, ab  a, aaaa
Complement:
L   * L
a, ba   , b, aa, ab, bb, aaa,
Courtesy Costas Busch - RPI
62
Reverse
Definition:
Examples:
L  {w : w  L}
R
R
ab, aab, baba  ba, baa, abab
R
L  {a b : n  0}
n n
L  {b a : n  0}
R
n n
Courtesy Costas Busch - RPI
63
Concatenation
Definition:
Example:
L1L2  xy : x  L1, y  L2 
a, ab, bab, aa
 ab, aaa, abb, abaa, bab, baaa
Courtesy Costas Busch - RPI
64
Another Operation
Definition:
L 
LL

L
n
n
a, b  a, ba, ba, b 
aaa, aab, aba, abb, baa, bab, bba, bbb
3
0
Special case: L  
a , bba, aaa 0  
Courtesy Costas Busch - RPI
65
More Examples
L  {a b : n  0}
n n
L  {a b a b : n, m  0}
2
n n m m
2
aabbaaabbb L
Courtesy Costas Busch - RPI
66
Star-Closure (Kleene *)
Definition:
L*  L  L  L 
0
1
2
Example:
 ,

a, bb,



a, bb*  

aa
,
abb
,
bba
,
bbbb
,


aaa, aabb, abba, abbbb,
Courtesy Costas Busch - RPI
67
Positive Closure
Definition:

L  L  L 
1
2
 L *  
a, bb,


 
a, bb  aa, abb, bba, bbbb,

aaa, aabb, abba, abbbb,


Courtesy Costas Busch - RPI
68