Download COMPS263F - Open University of Hong Kong

COMPS263F
Li Tak Sing
Lectures 23-28
1
Counting and Discrete Probability
Permutations
The number of ways to arrange n different objects in a line: n(n1)(n-2)....1=n!
0!=1
This is because there is only one way in arranging 0 items in a line.
The number of permutations of n objects taken r at a time is given by
P(n,r)=n(n-1)...(n-r+1)
=
2
𝑛!
𝑛−𝑟 !
Permutations with repeated elements
If a bag contains some objects in which m1 are of type 1, m2 are
of type 2, .... mk are of type k. The number of permutation is:
𝑚1 + 𝑚2 +. . +𝑚𝑘 !
𝑚1 ! 𝑚2 ! … 𝑚𝑘 !
3
Example Worst-case lower bound for
comparison sorting
A list of n numbers can be arranged in n! ways. So a decision
tree for sorting n numbers must contain n! leaves. Since a
binary tree of depth d can contain at most 2d leaves, so we have:
n!≤2d
 log2n! ≤d
 log2n! is approximately nlog2n
nlog2n ≤d
4
People in a circle
 n different people are sitting arrange a circle table.
 the number of way to sit is (n-1)!.
5
Combination
 The number of ways of taking r objects from n different
objects so that the order is not import is:
𝑃(𝑛, 𝑟)
𝑛!
𝐶 𝑛, 𝑟 =
=
𝑟!
𝑟! 𝑛 − 𝑟 !
 C(n,r) is the coefficient of xr in the expansion of (x+1)n
𝑛
 C(n,r) can also be written in the form
.
𝑟
6
Example
Suppose we want to build a code to represent 29 objects in
which each object is represented as a binary string length n,
which consists of k 0's and m 1's and n=k+m. Find n, k and m,
where n has the smallest possible value. (n=7,k=4, m=3)
7
Bag combinations
 Now we want to know the number of way to form a bag of k
elements from a set of n elements.
 This can be though of as arranging n+k-1 balls so that k of
them are white balls and n-1 of them are black balls. The n-1
black balls divides the k white balls into n groups. We can
then consider the first group as the elements from the 1st
element of the set, the second from the second element and
𝑛+𝑘−1 !
𝑛+𝑘−1
so on. So the number of ways=
=
.
𝑛−1 !𝑘!
𝑘
8
Example
In how many ways can four coins be selected from a bags of
7
$1, $2, $5 and $10 coins. (
= 35)
3
2. n how many ways can a committee of 5 people be selected
6
from a group of doctors and nurses. (
= 6)
1
3. In how many ways can five people be selected from a
collection of Democrats, Republicans, and Independents?
Here we are choosing five-element bags from a set of three
characteristics {Democrat, republican, Independent}.
7
(
= 21)
2
1.
9
Discrete probability
The set of all possible outcomes of an experiment is called a sample
space or probability space.
The elements in a sample space are called sample points or simply
points.
Any subset of a sample space is called an event.
A probability distribution on a sample space S is an assignment of
probabilities to the points of S such that the sum of all the
probabilities is 1.
Le S={x1, x2, ...xn} be a sample space. A probability distribution P
on S is a function
P: S[0,1]
such that
P(x1)+P(x2)+...+P(xn)=1.
10
Discrete probability
The probability of an event E is denoted by
𝑃 𝐸 = 𝑥∈𝐸 𝑃(𝑥)
P(S)=1
P()=0
P(AB)=P(A)+P(B)-P(AB)
P(E')=1-P(E)
11
The birthday problem
 Given n people in a room, what is the probability that at least
two of the people have the same birthday (month and day)?
 Assume that a year has 365 days. So the sample space
consists of 365n elements.
 Now, we want to calculate the number of cases where no two
people have the same birthday.
 The first person has 365 choices, the second has 364 choices,
the last has 366-n choices.
12
The birthday problem
 The total number of choices are 365*364*...*(366-n).
 So the probability is 1 −
365×364×⋯×(366−𝑛)
365𝑛
 For 40 person, the probability is 0.891. So it is very likely to
have two persons having the same birthday.
13
Conditional probability
 P(A|B)=probability that event A would occur given that B
has
𝑃(𝐴∩𝐵)
occurred=
𝑃(𝐵)
 If E1, ...En form a partition of S, then
𝑃(𝐸𝑖 ∩ 𝐵)
𝑃 𝐸𝑖 𝐵 =
𝑃(𝐵)
𝑃 𝐵|𝐸𝑖 𝑃(𝐸𝑖 )
=
𝑃 𝐵|𝐸1 𝑃 𝐸1 + ⋯ + 𝑃 𝐵|𝐸𝑛 𝑃(𝐸𝑛 )
 If A and B are independent event, then
P(A)=P(A|B)=P(A|B'). This means that the occurrence of
B does not in any way affect the occurrence of A.
 Therefore P(A)P(B)=P(AB)
14
Repeated independent trials
 Let there be a game so that the probability of wining is p.
Then the game is played for n times. The probability of
𝑛 𝑟
winning exactly r times is:
𝑝 (1 − 𝑝) 𝑛−𝑟
𝑟
15
Example
A team has probability 2/3 of winning whenever it plays. Find each
of the following probabilities that the team will win.
1.
a)
b)
c)
A student is chosen at random from a class of 80 students that has 20
honor students, 30 athletes, and 40 that are neither honor students
nor athlets.
2.
a)
b)
c)
16
Exactly 4 out of 5 games.
At most 4 out 5 games.
Exactly 4 out of 5 games given that it has already won the first 2 games of a
5-game series.
What is the probability that the student selected is an athlete given that he
or she is an honors students.
What is the probability that the student selected is an honors student
given that he or she is an athlete?
Are the events "honors student" and "athlete" independent?
Solution
5
1. a)
4
5
b)15
3
c)
2
2.
2 4 1 1
3
3
2 5 1 0
3
3
2 2 1 1
3
3
S-all students,H-honor students, A-athletes.
Therefore |S|=80,|H|=20, |A|=30, |HA|=40,
|HA|=|H|+|A|-|HA|=10
𝑃(𝐴𝐻)
|𝐴𝐻|
10
1
a) 𝑃 𝐴 𝐻 =
=
= =
𝑃(𝐻)
17
|𝐻|
20
2
Solution
2.
b)𝑃
c) 𝑃
𝑃(𝐴𝐻)
|𝐴𝐻|
𝐻𝐴 =
=
𝑃(𝐴)
|𝐴|
|𝐻|
20
1
𝐻 =
= =
|𝑆|
80
4
=
10
30
=
therefore can see that 𝑃 𝐻 ≠ 𝑃 𝐻 𝐴
H and A are not independent.
18
1
3
Examples
1.
2.
19
A computer program uses one of three procedures for each
piece of input. The procedures are used with probabilities 1/3,
1/2, 1/6. Negative results are detected at rates of 10%, 20%,
and 30% by the three procedures, respectively. Suppose a
negative result is detected. Find the probabilities that each of
the procedures was used.
A commuter crosses one of three bridges, A, B, or C, to go
home from work, crossing A with probability 1/3, B with
probability 1/6, and C with probability 1/2. The commuter
arrives home by 6 p.m. 75%, 60% and 80% of the time by
crossing bridges A, B, and C, respectively. If the commuter
arrives home after 6 p.m., find the probability that bridge A was
used. Also find the probabilities for bridges B and C.
Expectation
 expectation = average behavior
 For example, in rolling a die, the probabilities of different outcomes are:
1: 1/6
2: 1/6
3: 1/6
4: 1/6
5: 1/6
6: 1/6
So if a die is rolled n times, roughly n/6 of them would be 1, n/6 of them
would𝑛be 2, etc.𝑛 We can 𝑛then calculate
of the
𝑛 the average
𝑛
𝑛 outcome:
1× +2× +3× +4× +5× +6×
6
6
6
6
6
6
1
1
1𝑛
1
1
1
𝑘
=1× +2× +3× +4× +5× +6×
6
6
6
6
6
6
=
𝑃 𝑥𝑟 𝑉(𝑥𝑟 )
(xr) 𝑟=1
is the value of the outcome.
20
Example
Find the expected score obtained by guessing all 100
questions of a true-false exam in which a correct answer is
worth 1 point and an incorrect answer is worth -1/2 point.
So would you suggest a student to do a random guess if
he/she does not know the answer?
2. In a simple rolling die game, if a player bets $1 for a
number 1 to 6, and if he wins, he will get 4 dollars. Is this
game a fair game?
1.
21
Average Performance of an Algorithm
 If in an algorithm, there are a number of possible inputs with different
probabilities, and each input may result in different number of
operations being executed, then we can compute the average number of
operations accordingly.
 Suppose the sample space is {I1, I2,...,Ik}, P(Ij) is the probability of the
input being Ij, V(Ij) is the number operations executed if the input is Ij.
Then, the average number of operations executed is: 𝑘𝑗=1 𝑃 𝐼𝑗 (𝑉𝑗 )
 To show that an algorithm A is optimal in the average case for some
problem, we need to specify a particular sample space and probability
distribution. Then we need to show that AvgA(n)AvgB(n) for all n>0
and for all algorithms B that solve the problem.
 However, it is difficult to find the optimal algorithm. So we usually just
find the best from known algorithms.
22
Example
 Analysis of sequential search
 To search an element X in an array L
The algorithm is:
i:=n;
while i1 and XL[i] do
i:=i-1
od
The even Ij, j=1,..,n represents the case when X is at position j. In+1
represents the case when X is not in L. So V(Ij)=n-j+1 for j=1,...,n.
V(In+1)=n.
Assume that q is the probability that X is in L. So P(Ij)=q/n for
j=1,...,n. P(In+1)=1-q.
AvgA(n)=E(V)=(n+(n-1)+...+1)(n/q)+(1-q)n=q(n+1)/2 + (1-q)n
23
Example
 So if we know that X is in L, then q=1, then
AvgA(n)=(n+1)/2. If X is not in L, then q=0, AvgA(n)=n.
24
Monte Carlo Method
 Sometimes it is difficult to find a formula to a problem. We
can use a statistical method called Monte Carlo method.
 For example, in finding the area of a shape, we can draw the
same on a piece of paper. Then, we can randomly put points
onto the paper. The percentage of points inside the shape can
then be used to estimate the area of the same.
 In estimating the average performance of an algorithm, we
need to have some random input and then find the average
number of operations accordingly.
25
Example
Assume that we have a sorted list of 15 elements, x1, x2,
..., x15. Calculate the average number of comparisons
made by a binary search algorithm to look for a key that
may or may not be in the list. Assume that the key has
probability 1/2 of being in the list and that each of the
events "key=xi" is equally likely for i=1..15.
2. Generalize the problem to find a formula for the average
number of comparisons used to look for a key in a sorted
list of size n=2k-1, where k is a natural number. Assume
that the key has probability p of being in the list and that
each of the events "key=xi" is equally likely for i=1,..,n.
1.
26
Chapter 11 Regular Languages and
Finite Automata
The regular Languages
Regular languages are constructed from the letters of an
alphabet by using the language operations of union,
concatenation, and closure.
, {}, and {a} are regular languages for all aA.
If L and M are regular languages, then the following languages
are also regular: LM, ML, and L*.
There can be many regular Languages.
27
The regular languages
For example: A={a,b}
The following are all regular languages of A:

{}
{a}
{b}
{a,b}
{aa,bb}
Actually, any finite set that contains elements of strings that contain
a, b are regular languages of A.
28
Regular Expressions
A regular language is often described by means of an algebraic
expression called a regular expression.
, , and a are regular expressions for all aA.
If R and S are regular expressions, then the following
expressions are also regular: (R), R+S, RS, and R*
For example, A={a,b}
The following are regular expressions of A:
, , a, b, +b, b*, a+(ba), (a+b)a, ab*, a*+b*.
29
Regular Expressions
To avoid using too many parentheses, we assume that the
operations have the following hierarchy:
 * highest (do it first)
 .
 + lowest (do it last).
So a+ba* is equivalent to (a+(b(a*))).
In addition, we can omit . There, ab can be written as ab.
30
Relations between regular expressions
and regular languages
 L()=
 L()={}
 L(a)={a} for each aA
 L(R+S)=L(R)L(S)
 L(RS)=L(R)L(S)
 L(R*)=L(R)*
31
Example
L(a*+b*) = L(a*)L(b*)
=L(a)* L(b)*
={a}*{b}*
={, a, a2, a3,....} {, b, b2, b3,....}
={, a, b, a2, b2, a3, b3, ..... }
32
Problems
Find a regular expression to describe each of the following
languages.
1. {a,b,c}
2. {aa,ab,ac}
3. {a, b, ab, ba, abb, baa, ...., abn, ban,...}
4. {a, aaa, aaaaa, ...., a2n+1}
5. {, a, abb, abbbb, ..., ab2n, ... }
6. {, a, b, c, aa, bb, cc, ..., an, bn, cn, ....}
7. {ambcn| m,n N}
33
Equality of two regular expressions
 Different regular expressions may be related to the same




34
regular languages.
L(a+b)=L(b+a)={a,b}
Two regular expressions are said to be equal if they
correspond to the same regular language.
S=R if L(S)=L(R)
So we can say that a+b=b+a
(a+b)+(b+a)=a+b
abba
Properties of Regular Expressions
1. (+ properties)
R+T=T+R
R+=+R=R
R+R=R
(R+S)+T=R+(S+T)
2. ( properties)
R=R=
R=R=R
(RS)T=R(ST)
3. (Distributive properties)
R(S+T)=RS+RT
(S+T)R=SR+TR
35
Properties of Regular Expressions
(Closure properties)
4. *=*=.
5. R*=R*R*=(R*)*=R+R*.
R*=+R*=(+R)*=( +R)R*=+RR*
R*=(R+...+Rk)* for any k1,
R*=+R+...+Rk-1+RkR* for any k1.
6. R*R=RR*
7. (R+S)*=(R*+S*)*=(R*S*)*=(R*S)*R*=R*(SR*)*
8. R(SR)*=(RS)*R
9. (R*S)*= +(R+S)*S,
(RS*)*=+R(R+S)*
36
Problems
1. Find a regular expression for each of the following
languages over the alphabet {a,b}.
Strings with even length.
b. Strings whose length is a multiple of 3.
c. Strings containing the substring aba.
d. Strings with an odd number of a's.
a.
2. Find a regular expression over the alphabet {0,1} to
describe the set of all binary numerals without leading
zeros (except 0 itself). So the language is the set {0, 1,
10, 11, 100, 101, 110, 111,...}
37
Example
Write down the regular expressions for the following regular
languages of the alphabet A={a,b,c} .
(a) {abc}
(b) {abc, acb}
(c) {aibjck| i, j, k  N}
(d) {ab, bc, ca, ba, cb, ac}
(e) {aa, bb, cc}
38
Solution
(a) abc
(b) a(bc+cb)
(c) a*b*c*
(d) a(b+c)+b(c+a)+c(a+b)
(e) aa+bb+cc
39
Example
Write down the regular languages for the following regular
expressions of alphabet A={a,b,c}
(a) (a+b)b*
(b) abc*
(c) a+c*
(d) b(ab)*
(e) (a+b)*
40
Solution
(a) {abk, bbk| kN}
(b) {abck|kN}
(c) {a, bk| kN}
(d) {b(ab)k|kN}
(e) {}{r1r2…rn| r1, r2, …rn{a,b}}
41
Finite Automata
 This is a way of determining whether a string belongs to a
language.
 Deterministic Finite Automata
 A deterministic finite automaton over a finite alphabet A can be
thought of as a finite directed graph with the property that each
node emits one labeled edge for each distinct element of A.
 The nodes are called states.
 There is a special state called the start or initial state, and there
is a set of states called final states.
 It can be abbreviated as DFA.
42
Deterministic Finite Automata
 The following figure represents a DFA over the alphabet
A={a,b}
 Start represents the start state.
 Final state is presented by a double circle.
 Each state emits exactly two arrows, one labeled with a and
one labeled with b.
43
The execution of a DFA
 The movement along an edge from one state to another is
called a state transition, and the input letter corresponding to
the label on the edge is consumed by the state transition.
 A DFA accepts a string w in A* if the execution path ends in a
final state. Otherwise, the DFA rejects w. The language of a
DFA is the set of strings that it accepts.
 For example, the above DFA accepts the following strings:
 abbaaba
 baaabab
 bbaa
 The DFA rejects the following strings:
 abbbb
 baaa
44
Examples
 (a+b)*
 a(a+b)*
 State 2 acts as an error state because every state need to
emits two arrows.
45
Some simple rule:
 a+b
a
b
a
 a*b
b
b
 ab*
a
46
Problems
Draw a DFA to recognize the language of each of the following
regular expressions.
 a+ba*.
 a*b.
 ba*.
47
Nondeterministic Finite Automata
 Similar to a DFA, but each state emits zero or more edges,
and each edge is labeled either with a letter from A or with
.
 A state may emit more than one edge labeled with the same
letter.
48
The execution of an NFA
 Similar to a DFA except that an edge labeled with the empty
string is traversed without consuming an input letter.
 Since a state may emit an edge labeled with  or it may emit
more than one edge labeled with the same letter, there may
be alternative paths to acceptance, which is why it is
nondeterministic.
49
Examples
 a*a
 aa*
50
Regular Expression to Finite Automaton
Given a regular expression, we start the algorithm with a
machine that has a start state, a single final state, and an edge
labeled with the given regular expression as follows:
Now transform this machine into a DFA or an NFA by applying
the following rules until all edges are labeled with either a
letter or A :
1. If an edge is labeled with , then erase the edge.
2. Transform any diagram like
into the diagram
51
Regular Expression to Finite Automaton
3.Transform any diagram like
into the diagram
4. Transform any diagram like
into the diagram
52
Example
To construct an NFA for a* + ab, we'll start with the diagram
Next we apply rule 2 to obtain the following NFA:
Next we'll apply rule 4 to a* to obtain the following NFA:
53
Example
Finally, we apply rule 3 to ab to obtain the desired NFA for a*
+ ab:
54
Example
Draw a NFA to recognize the language of each of the following
regular expressions:
1. ab*+b(a+ba)*b
2. (a+b)*ba*
55
Example
Given the following regular expressions and strings, match each string to the corresponding regular
expression. Note that a string can correspond to more than one regular expression.
Regular expressions:
56
(a)
(b)
(c)
(d)
(e)
(a+b)c*(c+a)
a*(b+a)c
ac(b+a)
(a+b)(b+c)*
(ab+bc)*
Strings
(i)
(ii)
(iii)
(iv)
(v)
abc
bbb
abbc
acca
aaaa
_____________
_____________
_____________
_____________
_____________
Solution
(i) b,d
(ii) d
(iii) d,e
(iv) a
(v) None
57
Example
Write down the DFAs of the following regular expressions.
(a)
(b)
58
a(b*+c*)
a(b+c)*
Solution
(i)
b
a
3
b
start
1
a
2
a, c
5
c
c
a, b
4
b, c
(ii)
b, c
start
1
a
3
59
2
a
b, c
a, b, c
a, b, c
Example
Write down the NFAs of the following regular expressions.
(a)
(b)
60
a(b*+c*)
a(b+c)*
Solution
(a)
b
3
Λ
Λ
start
a
s
2
f
c
Λ
Λ
4
(b)
b,c
start
61
s
a
2
Λ
3
Λ
f
Finite Automaton to Regular Expression
Assume that we have a DFA or an NFA. Perform the following
steps:
1. Create a new start state s, and draw a new edge labeled with
A from s to the original start state.
2. Create a new final state f, and draw new edges labeled with
A from all the original final states to f
3. For each pair of states i and j that have more than one edge
from i to j, replace all the edges from i to j by a single edge
labeled with the regular-expression formed by the sum of the
labels on each of the edges from i to j.
62
Finite Automaton to Regular Expression
4. Construct a sequence of new machines by eliminating one state at a
time until the only states remaining are s and f. As each state is eliminated,
a new machine is constructed from the previous machine as follows:
Eliminate State k
For convenience we'll let old(i, j) denote the label on edge (i, j) of the
current machine. If there is no edge (i, j), then set old(i, j) =. Now for
each pair of edges (i, k) and (k, j), where i k and j  k, calculate a new edge
label, new(i, j), as follows:
new(i, j) = old(i, j) + old(i, k) old(k, k)* old{k, j).
For all other edges (i, j) where i  k and j  k, set
new(i, j) = old(i,j).
63
Finite Automaton to Regular Expression
The states of the new machine are those of the current machine
with state k eliminated. The edges of the new machine are the
edges (i, j) for which label new(i, j) has been calculated.
Now s and f are the two remaining states. If there is an edge (s,
f), then the regular expression new(s, f) represents the language
of the original automaton. If there is no edge (s, f), then the
language of the original automaton is empty, which is signified
by the regular expression .
64
Example
Let's walk through a simple example to demonstrate the
algorithm. Suppose we start with the DFA in the above figure.
The first three steps transform this machine into the following
machine, where s is the start state and f is the final state:
65
Example
Now let's eliminate the states 0, 1, and 2. We can eliminate state 2 without any work because there are
no paths passing through state 2 between states that are adjacent to state 2. In other words, new(i, j) =
old(i, j) for each edge (i, j), where i 2 and j 2. This gives us the machine
Now we'll eliminate state 0 from this machine by adding a new edge (s, 1) that is labeled with the
following regular expression:
new (s, 1) = old (s, 1) + old (s, 0) old (0,0)* old (0,1)
=  + *a = a.
Therefore, we delete state 0 and add the new edge (s, 1) labeled with a to obtain the following
machine:
66
Example
Next we eliminate state 1 in the same way by adding a new edge (s,
f) labeled with the following regular expression:
new (s, f) = old (s, f) + old (s, 1) old (1,1)* old (1, f)
=  + a(a + b)*  = a(a + b)*.
Therefore, we delete state 1 and label the edge (s, f) with a(a + b)*
to obtain the following machine:
This machine terminates the algorithm. The label a(a + b)* on edge
(s, f) is the regular expression representing the regular language of
the original DFA.
67
Example
We'll verify that the regular expression (a + b)*abb does indeed
represent the regular language accepted by the DFA in the
following figure.
68
Example
We start the algorithm by attaching start state s and final state f to the DFA in to
obtain the following machine:
Now we need to eliminate the internal states. As we construct new edge labels,
we'll simplify the regular expressions as we go. First we'll eliminate the state 0. To
eliminate state 0, we construct the following new edges:
new (s, 1) =  + b* a = b*a,
new (3,1) = a + bb*a = ( + bb*) a = b*a.
These new edges eliminate state 0 and we obtain the following machine:
69
Example
The states can be eliminated in any order. For example, we'll eliminate state 3
next, which forces us to create the following new edges:
new (2, f) =  + b * = b,
new (2,1) = a + b *b*a = a + bb*a = ( + bb*) a = b*a.
With state 3 eliminated, we obtain the following machine:
Now eliminate state 2, which forces us to create the following new edges:
new (1, f) =  + b*b = bb,
new (1,1) = a + b*b*a = a + bb*a = ( + bb*) a = b*a. With state 2
eliminated, we obtain the following machine:
70
Finally, we remove state 1 by creating a new edge,
new (s,f) =  + b*a(b*a)*bb
= b*(ab*)*abb
= (a + b)* abb
With state 1 eliminated, we obtain the desired regular
expression for the DFA .
71
Problem
Given the following NFA.
Perform one step in the transformation of the NFA to a regular
expression by removing state 1 and then drawing a picture of
the machine that results. Don't proceed further. Just remove
state 1.
72
Finite Automata as Output Devices
 The automata that we've discussed so far have only a limited
output capability to indicate the acceptance or rejection of an
input string. We want to introduce two classic models for
finite automata that have output capability. We'll consider
machines that transform input strings into output strings.
These machines are like DFAs, except that we associate an
output symbol with each state or with each state transition,
and there are no final states because we are not interested in
acceptance or rejection.
73
Mealy and Moore Machines
 The first model, invented by Mealy [1955], is called a Mealy
machine. It associates an output letter with each transition.
For example, if the output associated with the edge labeled
with the letter a is x, we'll write a/x on that edge. A state
transition for a Mealy machine can be pictured as follows:
 In a Mealy machine, an output always takes place during a
transition of states.
74
Moore Machines
 The second model, invented by Moore [1956], is called a
Moore machine. It associates an output letter with each state.
For example, if the output associated with state i is x, we'll
always write i/x inside the state circle. A typical state
transition for a Moore machine can be pictured as follows:
 Each time a state is entered, an output takes place. So the
first output always occurs as soon as the machine is started.
75
An example
Let's look at an example problem, which we'll solve both with a
Mealy machine and with a Moore machine. Suppose we want to
compute the number of substrings of the form
bab
that occur in an arbitrary input string over the alphabet {a, b}. For
example, there are three such substrings in the string
abababaababb.
A Mealy machine to solve this problem is given by the following
graph:
76
For example, the output of this Mealy machine for the sample string
abababaababb is 000101000010.
The problem can also be solved by the following Moore machine:
For example, the output of this Moore machine for the sample string
abababaababb is 000101000010.
We can count the number of l's in the output string to obtain the number
of occurrences of the substring bab.
From a practical point of view we might wish to let some output symbol
mean that no output takes place. For example, if we replaced the output
symbol 0 with  in the preceding machines, then the output for the sample
string would be the string 111.
77
Example
The Successor for Binary Numbers
Suppose we represent a natural number in the form of a binary string. To compute the
successor, we need to add 1. Using the standard addition algorithm, which involves
carrying, we can write down a Mealy machine to do the job. Since addition starts on the
right end of the string, we'll assume that the input is the binary representation in reverse
order.
We must consider the special case when a natural number has the form 2k - 1, which is
represented by a string of k l's. Thus when 1 is added to 2k - 1, we get 2k, which is
represented by a string of length k + 1. In this case, our machine will output a string of k
0's, which we will interpret as the number 2k. With this assumption the Mealy machine
looks like the following, where state 1 is the carry state and state 2 is the no-carry state:
78
Example
For example, let's find the successor of 13. The binary
representation of 13 is 1101. So we take its reverse, which is
1011, as input. The sequence of states for this input is 1, 2, 2, 2,
which gives the output string 0111. The reverse of this string is
1110, which is the binary representation of 14.
79
A Simple Vending Machine
Suppose we have a simple vending machine that allows the user
to pick from two 10-cent items A and B. To simplify things, the
slot will accept only dimes. There are four inputs to the
machine: d (dime), a (select item A), b (select item B), and r
(return coins). The outputs will be n (do nothing), A (vend item
A), B (vend item B), and d (dime). A Mealy machine to model
this simple vending machine can be pictured as follows:
80
Representing and Executing Finite
Automata
 When a DFA has an edge from i to j labeled with a, we can
denote it by writing T(i,a)=j.
 T(i,as)=T(T(i,a),s)
 Consider T(i,w), where i is the start state of a DFA and w is a
string. If the result state is a final state, then w is accepted.
Otherwise, w is rejected.
 T is known as the transition function.
81
Transition tables
The table containing the values of the transition function is a
transition table.
consider the above DFA. We have shown that this DFA
recognizes the language of (a+b)*abb over the alphabet
A={a,b}.
The transition table is:
start
final
82
T
0
1
2
3
a
1
1
1
1
b
0
2
3
0
Transition table
An example to verify whether the string abb belongs to the
langauge.
T(0, abb)=T(T(T(0,a),b),b)
=T(T(1,b),b)
=T(2,b)
=3
since 3 is the final state, the string is accepted.
83
Representations for NFAs
The transition tables for NFAs are different from those of DFAs in that
given a state and a letter, the next state can have more than one choice.
When the current state is 1 and the next letter is b, the next state can be 2
or 3. So we have: T(1,b)={2,3}. On the other hand, when the current
state is 1 and the next letter is b, then it cannot go anywhere. We have
T(1,a)=. In fact, we would have the following transition table:
start
final
84
T
0
1
2
3
a
{1}

{2,3}

b

{2,3}



{2}



Example
Given the following transition table, draw the corresponding
NFA:
start
final
85
T
0
1
2
3
a
{1}
{1,3}
{2,3}

b
{1}
{2}

{2}

{2}



Transforming an NFA into a DFA
 Each state of a DFA corresponds to a subset of NFA states.
 That is why it is called subset construction.
The followings are used to construct these subsets:
 The sets of states that occur in the NFA transition table.
 Certain sets of NFA states that are reachable by traversing 
edges.
 We need to define Lambda Closure:
86
Definition of Lambda Closure
If s is an NFA state, then the lambda closure of s, denoted (s), is
the set of states that can be reached from s by traversing zero or
more  edges. We can define  (s) inductively as follows for
any state s in an NFA.
 s  (s).
 If p   (s) and there is a  edge from p to q, then q   (s).
87
Constructing Lambda Closures
Lambda closures are easy to construct. For example, suppose
we have the following NFA, which we've described as a graph
as well as a table.
start
final
88
TN
a
b

0


{1}
1
{2,3}


2

{3}
{1}
3
{4}

{2,4}
4



Lambda closures
The lambda closures for the five states of the NFA are as
follows:
 (0)={0,1},
 (1)={1},
 (2)={1,2},
 (3)={1,2,3,4}
 (4)={4}.
The rule regarding lambda closure
 (CD)= (C) (D)
 ({s1, s2, ...., sn})=(s1)  (s2) ..... (sn)
89
NFA to DFA Algorithm
Input: An NFA over alphabet A with transition function TV.
Output: A DFA over A with transition function TD that accepts the same
language as the NFA. The states of the DFA are represented as certain
subsets of NFA states.
1.
The DFA start state is  (s), where s is the NFA start state. Now
perform
Step 2 for this DFA start state.
2. If {s1, ..., sn} is a DFA state and a  A, then construct the following
DFA state as a DFA table entry in either of two ways:
TD({s1,... ,sn},a)
= (TN (s1, a) ....... TN (sn, a))
(closure of union)
= ( (TN (s1, a)) ) ....... ( (TN (sn, a)) (union of closure).
Repeat Step 2 for each new DFA state constructed in this way.
3.
90
A DFA state is final if one of its elements is an NFA final state.
Steps in constructing the DFA of the
above figure
start state=(0)={0,1}
TD({0,1},a`)=(TN(0,a)) (TN(1,a))
=() ({2,3})
=(2) (3)
={1,2}{1,2,3,4}
start
TD
{0,1}
a
{1,2,3,4}
b

There are two new state, one is {1,2,3,4}, the other is . We
there have the following table:
start
final
91
TD
{0,1}
{1,2,3,4}

a
{1,2,3,4}
{1,2,3,4}

b

{1,2,3,4}

Steps in constructing the DFA of the
above figure
Then, we can replace {0,1} by state 0, {1,2,3,4} by state 1, 
by state 2. So the transition table becomes:
start
final
TD
0
1
2
a
1
1
2
b
2
1
2
Which is of course corresponds to the following DFA:
92
Example
Construct a DFA table for the following NFA:
start
final
93
T
0
1
2
a

{2}

b
{1,2}

{2}

{1}

{1}
Example
Suppose we are given the following NFA over the alphabet
{a,b}:
(i)
Find a regular expression for the language accepted by
the NFA.
(ii)
Write down the transition table for the NFA.
(iii) Transform the NFA into a DFA.
(iv) Draw a picture of the resulting DFA.
94
Example
Consider the regular expression
a*(a+ba)b*
(i)
Draw the diagram for the NFA of the expression.
(ii)
Write down the transition table of the expression
including the lambda closure of the states.
(iii) Write down the transition table of the DFA of the
expression.
(iv) Draw the diagram of the DFA of the expression.
95