Download + n - 朝陽科技大學

演算法概論 Introduction to Algorithms
Department of Computer Science and Information Engineering,
Chaoyang University of Technology
朝陽科技大學資工系
Speaker: Fuw-Yi Yang 楊伏夷
伏夷非征番,
道德經 察政章(Chapter 58) 伏者潛藏也
道紀章(Chapter 14) 道無形象, 視之不可見者曰夷
Fuw-Yi Yang
1
Reference:
K. H. Rosen, Discrete Mathematics and Its Applications, 5th ed.
J. A. Dossey, A. D. Otto, L. E. Spence and C. V. Eynden, Discrete
Mathematics, 4th ed.
R.1 Introduction
R.2 Recurrence Relations
R.3 Modeling with Recurrence Relations
R.4 Solving Recurrence Relations
R.5 First-Order Linear Difference Equation
R.6 Divide-and-Conquer Algorithms and Recurrence Relations
R.7 Prune-and-Search
Fuw-Yi Yang
2
R.1 Introduction
The number of a bacteria in a colony doubles every hour. If a colony
begins with five bacteria, how many will be present in n hours?
To solve this problem, let an be the number of bacteria at the end of n
hours. Since the number of bacteria doubles every hours, the
relationship an = 2an-1 holds whenever n is a positive integer.
This relationship, together with the initial condition a0 = 5,
uniquely determines an for all nonnegative integers n.
Solution: 5, 10, 20, 40, …
Fuw-Yi Yang
3
R.2 Recurrence Relations
Definition R.2.1. A recurrence relation for the sequence {an } is an
equation that expresses an in terms of one or more of the previous
terms of the sequence, namely, a0 , a1, …, an-1, for all integers n with
n ≧n0, where n0 is a nonnegative integer.
A sequence is called a solution of a recurrence relation if its terms
satisfy the recurrence relation.
Fuw-Yi Yang
4
R.2 Recurrence Relations
Connection between recursion and recurrence relations
Recursive algorithms provide the solution of a problem of size n in
terms of the solutions of one or more instances of the same problem of
a smaller size.
Consequently, when we analyze the complexity of a recursive
algorithm, we obtain a recurrence relation that expresses the
operations required to solve a problem of size n in terms of the
number of operations required to solve the problem for one or more
instances of smaller size.
Fuw-Yi Yang
5
R.2 Recurrence Relations
Example R.2.1
Let {an} be a sequence that satisfies the recurrence relation an =
an-1 – an-2 for n = 2, 3, 4,…, and suppose that a0 = 3 and a1 = 5.
What are a2 and a3?
Solution: a2 = a1 – a0 = 5 – 3 = 2,
a3 = a2 – a1 = 2 – 5 = -3
Fuw-Yi Yang
6
R.2 Recurrence Relations
Example R.2.2
Determine whether the sequence {an} is a solution of the
recurrence relation an = 2an-1 – an-2 for n = 2, 3, 4,…, where an = 3n
for every nonnegative integer n.
Answer the same question where an = 2n and where an = 5.
Solution: suppose that an = 3n for every nonnegative integer n.
Then for n ≧ 2, we have
an = 2an-1 – an-2 = 2(3(n - 1)) - 3(n - 2) = 3n.
Therefore, the sequence {an} is a solution of the recurrence
relation.
Fuw-Yi Yang
7
R.2 Recurrence Relations
Example R.2.3
Determine whether the sequence {an} is a solution of the
recurrence relation an = 2an-1 – an-2 for n = 2, 3, 4,…, where an = 3n
for every nonnegative integer n.
Answer the same question where an = 2n and where an = 5.
Solution: suppose that an =2n for every nonnegative integer n.
Then for n > 2, we have
an = 2*2n-1 – 2n-2 = 2n – 2n-2 ≠ 2n.
Therefore, the sequence {an} is not a solution of the
recurrence relation.
Fuw-Yi Yang
8
R.2 Recurrence Relations
Example R.2.4
Determine whether the sequence {an} is a solution of the
recurrence relation an = 2an-1 – an-2 for n = 2, 3, 4,…, where an = 3n
for every nonnegative integer n.
Answer the same question where an = 2n and where an = 5.
Solution: suppose that an = 5 for every nonnegative integer n.
Then for n ≧ 2, we have
an = 2*5 – 5 = 5.
Therefore, the sequence {an} is a solution of the recurrence
relation.
Fuw-Yi Yang
9
R.3 Modeling with Recurrence Relations
We can use recurrence relations to model a wide variety of problems,
such as finding compound interest, counting rabbits on an island,
determining the number of moves in the Tower of Hanoi puzzle, and
counting bit strings with certain properties.
Fuw-Yi Yang
10
R.3 Modeling with Recurrence Relations
Example R.3.1 Compound Interest.
Suppose that a person deposit $10000 in a saving account at a bank
yielding 11% per year with interest compounded annually. How much
will be in the account after 30 years?
Solution: Let pn denote the amount in the account after n years.
Then we see that the sequence {pn} satisfies the recurrence relation
Fuw-Yi Yang
11
R.3 Modeling with Recurrence Relations
Solution: Let pn denote the amount in the account after n years.
Then we see that the sequence {pn} satisfies the recurrence relation
pn = pn-1 + 0.11pn-1 = (1.11)pn-1.
The initial condition is p0 = 10000.
p1 = (1.11) p0
p2 = (1.11) p1 = (1.11)2 p0
…
pn = (1.11) pn-1 = (1.11)np0.
Thus p30 = (1.11)30
p30 = 228922.97
Fuw-Yi Yang
12
R.3 Modeling with Recurrence Relations
Example R.3.2 Rabbits and the Fibonacci Numbers.
Consider this problem, which was originally posed by Leonardo di
Pisa, also known as Fibonacci, in the thirteenth century in his book
Liber abaci.
A young pair of rabbits (one of each sex) is placed on an island. A
pair of rabbits does not breed until they are 2 months old. After they
are 2 months old, each pair of rabbits produces another pair each
month. Find a recurrence relation for the number of pairs of rabbits on
the island after n months, assuming that no rabbits ever die.
Solution: Let fn denote the number of pairs of rabbits after n months.
Fuw-Yi Yang
13
R.3 Modeling with Recurrence Relations
Solution: Let fn denote the number of pairs of rabbits after n months.
At the end of the first month, the number of pairs of rabbits on the
island is f1 = 1. At the end of the second month, the number of pairs of
rabbits on the island is f2 = 1. N.t.
Fuw-Yi Yang
14
R.3 Modeling with Recurrence Relations
Month
Reproducing
pairs fn-2
Young pairs
Total pairs
fn-1
fn
1
0
1
1
2
0
1
1
3
1
1
2
4
1
2
3
5
2
3
5
6
3
5
8
7
5
8
5
The sequence {fn} satisfies the recurrence relation fn = fn-1 + fn-2
Fuw-Yi Yang
15
R.3 Modeling with Recurrence Relations
Example R.3.3 Counting bit string of length n with no two
consecutive 0s.
Find a recurrence relation and give initial conditions for the number
of bit strings of length n that do not have two consecutive 0s. How
many such bit strings are there of length five?
Solution: Let an denote the number of bit strings of length n that do
not have two consecutive 0s.
Fuw-Yi Yang
16
R.3 Modeling with Recurrence Relations
Solution: Let an denote the number of bit strings of length n that do
not have two consecutive 0s.
To obtain a recurrence relation for {an}, note that by the sum rule,
the number of bit strings of length n that do not have two
consecutive 0s equals the number of such bit strings ending with a 0
plus the number of such bit strings ending with a 1.
Fuw-Yi Yang
17
R.3 Modeling with Recurrence Relations
an: the number of bit strings of length n that do not have two
consecutive 0s equals
1 the number of bit strings of length n that do not have two
consecutive 0s ending with a 1 plus
2 the number of bit strings of length n that do not have two
consecutive 0s ending with a 0.
an-1: the number of the first term (1) is precisely the number of bit
strings of length n-1 that do not have two consecutive 0s, i.e. an-1.
an-2: the number of the second term (2) is precisely the number of bit
strings of length n-2 that do not have two consecutive 0s, i.e. an-2.
Therefore, the recurrence relation an = an-1 + an-2 satisfies the sequence
{an}.
Fuw-Yi Yang
18
R.4 Solving Recurrence Relations
Definition R.4.1 A linear homogeneous recurrence relation of
degree k with constant coefficients is a recurrence relation of the
form
an = c1an-1+ c2an-2+…+ ckan-k,
where c1, c2, …, ck are real numbers, and ck ≠0.
Fuw-Yi Yang
19
R.4 Solving Recurrence Relations
Linear: since the right-hand side is a sum of multiples of the previous
terms of the sequence.
Homogeneous: since no terms occur that are not multiples of the ajs.
Constants: the coefficients of the terms of the sequence are all
constants, rather than functions that depend on n.
Degree: the degree is k because an is expressed in terms of the
previous k terms of the
sequence.
Fuw-Yi Yang
20
R.4 Solving Recurrence Relations
A sequence satisfying the recurrence relation in the definition is
uniquely determined by this recurrence relation and the k initial
conditions
a0 = C0, a1 = C1, … ,ak-1 = Ck-1.
Fuw-Yi Yang
21
R.4 Solving Recurrence Relations
Example R.4.1 some linear homogeneous recurrence relations:
degree 1: pn = (1.11) pn-1,
degree 2: fn = fn-1 + fn-2,
degree 5: an = an-5,
Example 2. some recurrence relations:
an = an-1 + (an-2)2 , not linear (square),
Hn = 2Hn-1 + 1 , not homogeneous (1),
Bn = nBn-1 , not constant coefficient (n).
Fuw-Yi Yang
22
R.4 Solving Recurrence Relations
Linear homogeneous recurrence relations are studied for two reasons.
First, they often occur in modeling of problems.
Second, they can be systematically solved.
Fuw-Yi Yang
23
R.4 Solving Recurrence Relations with Constant
Coefficients
The basic approach for solving linear homogeneous recurrence
relations is to look for solutions of the form an = rn, where r is a
constant.
Note that an = rn is a solution of the recurrence relation
an = c1an-1+ c2an-2+…+ ckan-k,
if and only if :
rn = c1rn-1+ c2rn-2+…+ ckrn-k.
Both sides dividing by rn-k, we obtain,
rk - c1rk-1 - c2rk-2 -…- ckrk-k = 0.
Fuw-Yi Yang
24
R.4 Solving Recurrence Relations with Constant
Coefficients
Note that an = rn is a solution of the recurrence relation
an = c1an-1+ c2an-2+…+ ckan-k,
if and only if :
rn = c1rn-1+ c2rn-2+…+ ckrn-k.
Both sides dividing by rn-k, we obtain,
rk - c1rk-1 - c2rk-2 -…- ckrk-k = 0.
Consequently, the sequence {an} with an = rn is a solution if and
only if r is a solution of this last equation, which is called the
characteristic equation of the recurrence relation.
The solutions of this equation are called the characteristic roots of
the recurrence relation.
Fuw-Yi Yang
25
R.4 Solving Recurrence Relations with Constant
Coefficients
Example R.4.2 What is the solution of the recurrence relation:
an = an-1 + 2an-2, with a0= 2 and a1 = 7 ?
Solution:
The characteristic equation of the recurrence relation is
r2 - r1 - 2 = 0.
The characteristic roots of the recurrence relation are r = 2 and r = -1.
Which one is the solution ?
Fuw-Yi Yang
26
R.4 Solving Recurrence Relations with Constant
Coefficients
Which one is the solution ?
With the same method of solving homogeneous differential
equations, we guess the solution may be the combination of both of
them. The followings show that.
Let the characteristic roots be r1 and r2 for a characteristic
equation r2 - c1r1 - c2 = 0.
The corresponding recurrence relation is an = c1an-1 + c2an-2. Also
A and B denote constant coefficients.
Fuw-Yi Yang
27
R.4 Solving Recurrence Relations with Constant
Coefficients
The corresponding recurrence relation is an = c1an-1 + c2an-2. Also
A and B denote constant coefficients.
If the combination of r1 and r2 is solution, then an = A(r1)n + B(r2)n.
That is: c1an-1 + c2an-2 = c1 (A(r1)n-1 + B(r2)n-1) + c2 (A(r1)n-2 + B(r2)n-2)
= A(r1)n-2(c1r1 + c2) + B(r2)n-2(c1r2 + c2)
= A(r1)n-2(r1)2 + B(r2)n-2(r2)2
= an.
Fuw-Yi Yang
28
R.4 Solving Recurrence Relations with Constant
Coefficients
Hence the sequence {an} is a solution to the recurrence relation iff
an = A(2)n + B(-1)n,
From the initial conditions, it follows that
a0 = 2 = A + B,
a1 = 7 = 2 – B.
Thus, A = 3 and B = -1.
Hence, the solution to the recurrence relation and initial conditions is
the sequence {an} with
an = 3*2n - (-1)n.
Fuw-Yi Yang
29
R.4 Solving Recurrence Relations with Constant
Coefficients
Theorem R.4.1
Let c1 and c2 be real numbers. Suppose that r2 - c1r - c2 = 0 has two
distinct roots r1 and r2. Then the sequence {an} is a solution of the
recurrence relation an = c1an-1 + c2an-2 iff an = A(r1)n + B(r2)n for n =
0, 1, 2, …, where A and B are constant coefficients.
Fuw-Yi Yang
30
R.4 Solving Recurrence Relations with Constant
Coefficients
Theorem R.4.2
Let c1 and c2 be real numbers. Suppose that r2 - c1r - c2 = 0 has only
one roots r0. Then the sequence {an} is a solution of the recurrence
relation an = c1an-1 + c2an-2 iff an = A(r0)n + Bn(r0)n for n = 0, 1, 2, …,
where A and B are constant coefficients.
Fuw-Yi Yang
31
R.4 Solving Recurrence Relations with Constant
Coefficients
Theorem R.4.3
Let c1, c2, …, ck be real numbers. Suppose that the characteristic
equation
rk - c1rk-1 - …- ck = 0 has k distinct roots r1, r2, …, rk.
Then the sequence {an} is a solution of the recurrence relation
an = c1an-1 + c2an-2 + … + ckan-k iff
an = A1(r1)n + A2 (r2)n + …+Ak (rk)n for n = 0, 1, 2, …, where A1,
A2, …, Ak are constant coefficients.
Fuw-Yi Yang
32
R.5 First Order Linear Difference Equation
with Constant Coefficient
The simplest type of recurrence relation give sn as a function of sn-1 for
n≧ 1. We call an equation of the form
sn = asn-1 + b,
where a and b are constants and a ≠ 0, a first order linear
difference equation with constant coefficients.
It is not difficult to derive estimates of the size of functions satisfy
such recurrence relations.
sn = a sn-1 + b
= a (a sn-2 + b) + b = a2sn-2 + ab + b
= a3sn-3 + a2b + ab + b
=…
= ans0 + an-1b + … + ab + b
Fuw-Yi Yang
33
R.5 First Order Linear Difference Equation
with Constant Coefficient
sn = a sn-1 + b
= ans0 + an-1b + … + ab + b
IF a = 1, then sn = s0 + nb
If a ≠ 1, sn = sn-1 + b
= ans0 + an-1b + … + ab + b
= ans0 + b(an - 1)/ (a - 1)
= an(s0 + b/(a - 1)) - b/(a - 1)
Fuw-Yi Yang
34
R.5 First Order Linear Difference Equation
with Constant Coefficient
Example R.5.1 Sequential Search Algorithm
This algorithm searches a list of n items a1, a2, …,an for a given
target value t. If t = ak for some index k, then the algorithm gives the
first such index k. Otherwise the algorithm gives k = 0.
Step 1 j = 1
Step 2 while j ≦n and aj ≠t
j=j+1
Step 3 if j ≦n then k = j
elsek k = 0
Fuw-Yi Yang
35
R.5 First Order Linear Difference Equation
with Constant Coefficient
To search a list of n items, we compare the target value to the first
item in the list and then, in the worst case, still have to search a list of
n-1 items. Thus we see that sn satisfies the recurrence relation
sn = sn-1 + 1
for for n≧ 1. Also, the initial condition s0 = 0.
sn = sn-1 + 1
= (sn-2 + 1) + 1
= s0 + n
=n+1
Therefore the complexity of this algorithm is O(n).
Fuw-Yi Yang
36
R.5 First Order Linear Difference Equation
with Constant Coefficient
Example R.5.2 Find a formula for sn if sn = 3 sn-1 -1 for n ≧1.
The initial condition is s0 = 2.
Solution:
sn = asn-1 + b
= an(s0 + b/(a - 1)) - b/(a - 1),
where a = 3 and b = -1. Therefore,
sn = 3n(2 - 1/2) + 1/2
=1/2(3n+1 +1).
The time complexity of sn is of O(3n).
Fuw-Yi Yang
37
R.5 First Order Linear Difference Equation
Example R.5.3 Bubble Sort
This algorithm places the numbers in the list a1, a2, …, an in nondecreasing order.
Step 1 for j = 1to n-1
// set beginning of sublist
Step 2
for k = n-1 to j by -1 // find the smallest element
Step 3
if ak+1 < ak interchange the values of ak+1 and ak
Step
endfor
Step endfor
Step 4 Print a1, a2, …, an
Fuw-Yi Yang
38
R.5 First Order Linear Difference Equation
In performing a bubble sort, the first iteration requires comparing
an-1 to an, then an-2 to an-1, and so forth until a2 is compared to a1.
After each such comparison, we interchange the values of ak and
ak+1, if necessary, so that ak≦ak+1. Thus when the first iteration is
complete, a1 is the smallest number in the original list.
A second iteration is then performed on the smaller list a2, a3,…,an
in order to make a2 the second-smallest number in the original list.
After n-1 such iterations, the original list will be non-decreasing
order. Thus the recurrence relation is bn = bn-1 + (n-1) with initial
condition b1 = 0. Find its complexity.
Fuw-Yi Yang
39
R.5 First Order Linear Difference Equation
Solution: The recurrence relation is bn = bn-1 + (n-1), for n ≧ 2
initial condition b1 = 0.
b1 = 0,
b2 = 0 + 1,
b3 = 0 + 1 + 2, we conjecture that
bn = 1 + 1 + 2 +…+ (n-1) = (n2 – n)/2
Thus bn is O(n2).
Fuw-Yi Yang
40
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations
Recurrence relations of the form f(n) = af(n/b) + g(n)
This form of recurrence relation arise in many differential situations.
It implies the concepts of divide and conquer.
It is possible to derive estimates of the size of functions satisfy such
recurrence relations. Suppose that f satisfies this recurrence relation
whenever n is divisible by b. Let n = bk, where k is a positive integer.
Then
f(n) = a f(n/b) + g(n)
= a (af((n/b)/b) + g(n/b)) + g(n) = a2f(n/b2) + ag(n/b) + g(n)
= a3f(n/b3) + a2g(n/b2) + ag(n/b) + g(n)
Fuw-Yi Yang
41
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations
f(n) = a f(n/b) + g(n)
= a (af((n/b)/b) + g(n/b)) + g(n) = a2f(n/b2) + ag(n/b) + g(n)
= a3f(n/b3) + a2g(n/b2) + ag(n/b) + g(n)
=…
= akf(n/bk) + ak-1g(n/bk-1) + … + ag(n/b) + g(n)
= akf(1) + ak-1g(n/bk-1) + … + ag(n/b) + g(n)
Fuw-Yi Yang
42
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations
Recurrence relations of the form f(n) = af(n/b) + g(n)
f(n) = a f(n/b) + g(n)
= akf(n/bk) + ak-1g(n/bk-1) + … + ag(n/b) + g(n)
= akf(1) + ak-1g(n/bk-1) + … + ag(n/b) + g(n)
Since n = bk, thus k = logbn.
We can use this equation for f(n) to estimate the size of functions
that satisfy divide and conquer relation.
Fuw-Yi Yang
43
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations
Theorem R.6.1
Let f be an increasing function that satisfies the recurrence
relation f(n) = af(n/b) + c.
Whenever n is divisible by b, where a ≧1, b is an integer greater
than 1, and c is a positive real number. Then
f(n) is O(nlog ba) if a > 1,
O(log n) if a = 1.
Furthermore, when n = bk, where k > 0,
f(n) = Anlogba + B,
where A = f(1) + c/(a - 1) and B = -c/(a - 1).
Proof:
Fuw-Yi Yang
44
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations
f(n) = af(n/b) + c
= akf(n/bk) + ak-1c + … + ac + c
= akf(1) + ak-1c + … + ac + c
∵n = bk
= akf(1) + (ak – 1)c / (a - 1)
If a = 1, then f(n) = f(1) + kc
if n is a power of b, n = bk, f(n) = f(1) + c logbn
if n is not a power of b, bk < n < bk+1,
f(n) = f(1) + c logbn < f(1) + c(k + 1)
= (f(1) + c) + ck
< (f(1) + c) + c logbn
for both cases, f(n) is O(log n).
Fuw-Yi Yang
45
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations
If a > 1, and n is a power of b, n = bk, it follows that
f(n) = akf(1) + c(ak – 1)/(a - 1)
= ak(f(1) + c/(a – 1)) - c/(a - 1)
= A n logba + B,
∵ak = alogbn = nlogba
where A = f(1) + c/(a – 1) , B = - c/(a - 1).
If a > 1, and n is not a power of b, by similar processing we obtain
the same result.
Thus f(n) is O(n logba).
End of proof
Fuw-Yi Yang
46
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations
Example R.6.1 Binary Search.
(In the recurrence relation f(n) = af(n/b) + c, a = 1 implies that the
Binary search is a method of Prune-and-Search)
procedure binary search (x: integer, a1, a2, …, an: increasing integers)
i := 1 {i is left endpoint of search interval}
j := n {j is right endpoint of search interval}
while i < j begin m := integer ((i + j) /2)
if x > am then i := m + 1
else j : = m
end
if x = ai then location := i
Fuw-Yi Yang
47
else location := 0
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations
The binary search algorithm reduces the search for an element in a
search sequence of size n to the binary search for this element in a
search sequence of size n/2, when n is even. Hence, the problem of
size n has been reduced to one problem of size n/2. Two comparisons
are needed to implement this reduction, one to determine which half of
the list to use and the other to determine whether any terms of the list
remain. Hence if f(n) is the number of comparisons required to search
for an element in a search sequence of size n, then f(n) = f(n/2) + 2,
when n is even. Find its time complexity.
Solution:
Fuw-Yi Yang
48
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations
Solution: The recurrence relation is f(n) = f(n/2) + 2,
i.e. a = 1, c = 2.
From algorithm, it can be seen f(1) = 1.
If n is a power of b, n = bk, it follows that
f(n) = f(1) + 2k
= 1 + 2 log bn.
Thus f(n) is O(log n).
Fuw-Yi Yang
49
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations ---Merge Sort
Example R.6.2 Merge Sort
Procedure mergesort (L = a1, a2, …, an)
If n > 1 then
m := int(n/2)
L1 := a1, a2, …, am
L2 := am+1, am+2, …, an
L = merge(mergesort(L1), mergesort(L2))
{L is sorted into elements in nondecreasing order}
Fuw-Yi Yang
50
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Merge Sort
Procedure merge (L1, L2: lists)
L := empty list
While L1 and L2 are both nonempty
remove smaller of first element of L1 and L2 from the list it is in
and put it at the left end of L
if removal of this element makes one list empty then remove all
elements form the other list and append them to L
Endwhile
{L is the merged list with elements in nondecreasing order}
Two sorted lists with m elements and n elements can be merged
into a sorted list using no more than m+n+1 comparisons.
Fuw-Yi Yang
51
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations – Merge Sort
Thus, the recurrence relation for merge sort algorithm is :
T(n) = 2T(n/2) + n.
Let n = 2k,
T(n) = 2T(n/2) + n
= 2kT(n/2k) + 2k-12 + … + 2(n/2) + n
= 2kT(1) + 2k-12 + … + 2(n/2) + n
= nT(1) + n log2n
∵n = 2k
Therefore T(n) is O(n log n).
Fuw-Yi Yang
52
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Example R.6.3 Quick Sort
Quick sort applies the divide-and-conquer paradigm.
Here is the three-step divide-and-conquer process for sorting a
typical subarray A[p...r], i.e. Divide, Conquer, and Combine.
Divide: partition the array A[p..r] into two subarrays (possibly empty)
A[p...q-1] and A[q+1...r] such that for each element of
A[p...q-1] is less than or equal to A[q], which is, in turn,
less than or equal to each element of A[q+1...r].
Compute the index q as part of this partitioning procedure.
Fuw-Yi Yang
53
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Divide: partition the array A[p..r] into two subarrays (possibly empty)
A[p...q-1] and A[q+1...r] such that for each element of
A[p...q-1] is less than or equal to A[q], which is, in turn,
less than or equal to each element of A[q+1...r].
Compute the index q as part of this partitioning procedure.
Conquer: Sort the two subarrays A[p...q-1] and A[q+1...r] by
recursive calls to quicksort.
Combine: Because the subarrays are already sorted, no work is
needed to combine them: the entire array A[p...r] is now
sorted.
Fuw-Yi Yang
54
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Procedure Quicksort(A, p, r)
If p < r then
q = Partition(A, p, r)
Quicksort(A, p, q-1)
Quicksort(A, q+1, r)
Fuw-Yi Yang
55
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Procedure Partition(A, p, r)
x = A[r]
i=p-1
for j = p to r – 1
if A[j]  x
i=i+1
swap(A[i], A[j])
swap(A[i+1], A[r])
return i + 1
Fuw-Yi Yang
56
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Fuw-Yi Yang
57
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Fuw-Yi Yang
58
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Worst-case partitioning
The worst-case behavior for quick sort occurs when the
partitioning routing produces one subproblem with n-1 elements and
one with 0 elements. Let us assume that this unbalanced partitioning
arises in each recursive call. The partitioning cost (n) time. Since
the recursive call on an array of size 0 just returns, T(0) = (1), and the
recurrence for the running time is:
T(n) = T(n-1) + T(0) + (n)
= T(n-1) + (n)  O(n2).
Fuw-Yi Yang
59
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Best-case partitioning
The best-case behavior for quick sort occurs when the partitioning
routing produces two subproblems, each of size no more than n/2 ,
since one is n/2 and one of size n/2 -1. The partitioning cost (n)
time and the recurrence for the running time is:
T(n) = 2T(n/2) + (n)
 O(n log n).
Fuw-Yi Yang
60
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Procedure Randomized-Partition(A, p, r)
i = Random(p, r)
swap(A[i], A[r])
return Partition(A, p, r)
Procedure Randomized-Quicksort(A, p, r)
If p < r then
q = Randomized-Partition(A, p, r)
Randomized-Quicksort(A, p, q-1)
Randomized-Quicksort(A, q+1, r)
Fuw-Yi Yang
61
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Average-case partitioning
1 n
Tavg (n)  cn   (Tavg ( j  1)  Tavg )
n j 1
2 n
 cn   Tavg ( j )
n j 1
 kn log en  O(n log n)
Fuw-Yi Yang
62
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Average-case partitioning
Prove that Tavg(n) ≦kn log n
Basis: n = 2 and k = 2(b+c), Note: Tavg(0) ≦b, Tavg(1) ≦b
Assumption: Tavg(n) ≦kn log n for 1 ≦n < m
Inductive step: for m we want to prove that Tavg(m) ≦km log m
Fuw-Yi Yang
63
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Average-case partitioning
Inductive step: for m we want to prove that Tavg(m) ≦km log m
4b 2 m 1
Tavg (m)  cm    Tavg ( j )
m m j 2
4b 2k m 1
 cm  
j ln( j )

m m j 2
4b 2k m
 cm  
x ln( x)dx

2
m m
4b 2k m 2 ln m m 2
 cm   [

]
m m
2
4
4b
km
 cm   km ln m 
 km ln m
m
2
Fuw-Yi Yang
64
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Average-case partitioning
duv  udv  vdu
 ln( x)dx  ln( x)  x  c
udv  duv  vdu
 udv  uv   vdu
dx
u  ln( x)  du 
x
dv  dx  v  x
 udv  uv   vdu
dx
 x ln( x)   x
x
 x ln( x)  x  c
Fuw-Yi Yang
65
R.6 Divide-and-Conquer Algorithms and Recurrence
Relations --- Quick Sort
Average-case partitioning
 x ln( x)dx  ?
dx
u  ln( x)  du 
x
x2
dv  xdx  v 
2
 udv  uv   vdu
x2
x 2 dx

ln( x )  
2
2 x
x2
x2

ln( x ) 
c
2
2
Fuw-Yi Yang
66
R.7 Prune-and-Search
The Selection Problem
The Selection Problem
A prune-and-Search algorithm to find the kth smallest element
Input: A set S of n elements
Output: the kth smallest element of S
1. If |S| ≦ 5, solve the problem by any method
2. Divide S into n/5 subsets.
3. Sort each subset of elements.
4. Recursively find the element x which is the medians of the median
of the n/5 subsets.
5. Partition S into S1, S2, and S3, which contains the elements less
than, equal to, and greater than x, respectively. (next page)
Fuw-Yi Yang
67
R.7 Prune-and-Search
The Selection Problem
5. Partition S into S1, S2, and S3, which contains the elements less
than, equal to, and greater than x, respectively.
6. If |S1| ≧ k, then discard S2 and S3 and solve the problem that selects
the kth smallest element from S1 during the next iteration;
else if |S1| + |S2| ≧ k, then x is the kth smallest element of S;
otherwise, let k = k - |S1| - |S2|. Solve the problem that selects
the kth smallest element from S3 during the next iteration.
Fuw-Yi Yang
68
R.7 Prune-and-Search
The Selection Problem—worst case
It follows that the number of elements greater than x is at least
3(1/2 n/5  - 2) ≧ 3n/10 – 6.
Similarly, at least 3n/10 – 6 elements are less than x.
Thus, in the worst case, step 6 calls algorithm recursively on at most
7n/10 + 6 elements.
s
s
s
s
s
s
s
s
x
b
b
b
b
b
b
b
b
b
b
b
Fuw-Yi Yang
69
R.7 Prune-and-Search
The Selection Problem—worst case
11, 10, 23, 40, 32, 25, 76, 45, 88, 28, 13, 32, 53, 29, 31, 19, 98, 54,
16, 42, 67, 73, 55, 62, 34, 88, 63, 17, 25, 74, 35, 67, 99, 48, 78, 93,
92, 97
10
13
16
25
34
17
35
92
11
29
19
28
55
25
48
93
23
31
42 (x)
45
62
63
67
97
32
32
54
76
67
74
78
40
53
98
88
73
88
99
23
63
31
67
42 (x)
97
45
62
Fuw-Yi Yang
70
R.7 Prune-and-Search
The Selection Problem
We make the assumption that any input of fewer than 140 elements
requires O(1) time. We can therefore obtain the recurrence
T(n) ≦ O(1) if n < 140,
T(n) ≦ T(n/5 ) + T(7n/10 + 6 ) + O(n) if n > 140.
T(n) ≦ c(n/5 ) + c(7n/10 + 6 ) + an
≦ cn/5 + c + 7cn/10 + 6c + an
= 9cn/10 + 7c + an
= cn + (-cn/10 + 7c + an)
Which is at most cn if (-cn/10 + 7c + an) ≦0.
To satisfy the inequality above, n > 140, c ≧20a.
Fuw-Yi Yang
71
R.7 Prune-and-Search
Example R.7.1 Binary Search.
procedure binary search (x: integer, a1, a2, …, an: increasing integers)
i := 1 {i is left endpoint of search interval}
j := n {j is right endpoint of search interval}
while i < j begin m := integer ((i + j) /2)
if x > am then i := m + 1
else j : = m
end
if x = ai then location := i
else location := 0
Fuw-Yi Yang
72
R.7 Prune-and-Search
The binary search algorithm reduces the search for an element in a
search sequence of size n to the binary search for this element in a
search sequence of size n/2, when n is even. Hence, the problem of
size n has been reduced to one problem of size n/2. Two comparisons
are needed to implement this reduction, one to determine which half of
the list to use and the other to determine whether any terms of the list
remain. Hence if f(n) is the number of comparisons required to search
for an element in a search sequence of size n, then f(n) = f(n/2) + 2,
when n is even. Find its time complexity.
Solution:
Fuw-Yi Yang
73
R.7 Prune-and-Search
Solution: The recurrence relation is f(n) = f(n/2) + 2,
i.e. a = 1, c = 2.
From algorithm, it can be seen f(1) = 1.
If n is a power of b, n = bk, it follows that
f(n) = f(1) + 2k
= 1 + 2 log bn.
Thus f(n) is O(log n).
Fuw-Yi Yang
74