Download Analysis of Recursive Algorithms

RAIK 283: Data Structures & Algorithms
Recursive Algorithm Analysis
Dr. Ying Lu
[email protected]
September 13, 2012
1
RAIK 283: Data Structures & Algorithms
 Giving
credit where credit is due:
• Most of the lecture notes are based on the slides
from the Textbook’s companion website
http://www.aw-bc.com/info/levitin
• Several slides are from Hsu Wen Jing of the
National University of Singapore
• I have modified them and added new slides
2
Example: a recursive algorithm

Algorithm:
if n=0 then F(n) := 1
else F(n) := F(n-1) * n
return F(n)
3
Example: another recursive algorithm
Algorithm F(n)
// Compute the nth Fibonacci number recursively
//Input: A nonnegative integer n
//Output: the nth Fibonacci number
if n  1 return n
else return F(n-1) + F(n-2)
4
Recurrence Relation

Recurrence Relation
5
Recurrence Relation

Recurrence Relation: an equation or inequality that
describes a function in terms of its value on smaller
inputs
6
Example: a recursive algorithm

Algorithm:
if n=0 then F(n) := 1
else F(n) := F(n-1) * n
return F(n)

What does this algorithm compute?
7
Example: a recursive algorithm

Algorithm:
if n=0 then F(n) := 1
else F(n) := F(n-1) * n
return F(n)

What does this algorithm compute?

What’s the basic operation of this algorithm?
8
Example: recursive evaluation of n !
Recursive definition of n!:
 Algorithm:

if n=0 then F(n) := 1
else F(n) := F(n-1) * n
return F(n)

M(n): number of multiplications to compute n!
with this recursive algorithm
9
Example: recursive evaluation of n !
Recursive definition of n!:
 Algorithm:

if n=0 then F(n) := 1
else F(n) := F(n-1) * n
return F(n)

M(n): number of multiplications to compute n!
with this recursive algorithm

Could we establish a recurrence relation for
deriving M(n)?
10
Example: recursive evaluation of n !
Definition: n ! = 1*2*…*(n-1)*n
 Recursive definition of n!:
 Algorithm:

if n=0 then F(n) := 1
else F(n) := F(n-1) * n
return F(n)
M(n) = M(n-1) + 1
 Initial Condition: M(0) = ?

11
Example: recursive evaluation of n !
 Recursive
definition of n!:
 Algorithm:
if n=0 then F(n) := 1
else F(n) := F(n-1) * n
return F(n)
 M(n)
= M(n-1) + 1
 Initial condition: M(0) = 0
 Explicit formula for M(n) in terms of n
only?
12
Time efficiency of recursive algorithms
Steps in analysis of recursive algorithms:



Decide on parameter n indicating input size
Identify algorithm’s basic operation
Determine worst, average, and best case for inputs of size n

Set up a recurrence relation and initial condition(s) for
C(n)-the number of times the basic operation will be
executed for an input of size n

Solve the recurrence to obtain a closed form or
determine the order of growth of the solution (see
Appendix B)
13
EXAMPLE: tower of hanoi

Problem:
• Given three pegs (A, B, C) and n disks of different sizes
• Initially, all the disks are on peg A in order of size, the
largest on the bottom and the smallest on top
• The goal is to move all the disks to peg C using peg B as an
auxiliary
• Only 1 disk can be moved at a time, and a larger disk cannot
be placed on top of a smaller one
B
n disks
A
C
14
EXAMPLE: tower of hanoi

Design a recursive algorithm to solve this problem:
• Given three pegs (A, B, C) and n disks of different sizes
• Initially, all the disks are on peg A in order of size, the largest
on the bottom and the smallest on top
• The goal is to move all the disks to peg C using peg B as an
auxiliary
• Only 1 disk can be moved at a time, and a larger disk cannot
be placed on top of a smaller one
B
n disks
A
C
15
EXAMPLE: tower of hanoi

Step 1: Solve simple case when n<=1?
Just trivial
B
A
B
C
A
C
Move(A, C)
16
EXAMPLE: tower of hanoi

Step 2: Assume that a smaller instance can be
solved, i.e. can move n-1 disks. Then?
B
B
A
A
C
C
B
A
C
17
EXAMPLE: tower of hanoi
B
B
A
A
C
C
B
A
C
B
18
A
C
EXAMPLE: tower of hanoi
B
B
A
A
C
C
B
TOWER(n, A, B, C)
A
C
B
19
A
C
EXAMPLE: tower of hanoi
B
TOWER(n-1, A, C, B)
B
A
A
C
C
B
Move(A, C)
TOWER(n, A, B, C)
A
C
B
TOWER(n-1, B, A, C)
20
A
C
EXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
if n<1 return;
TOWER(n-1, A, C, B);
Move(A, C);
TOWER(n-1, B, A, C)
}
21
EXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
if n<1 return;
TOWER(n-1, A, C, B);
Move(A, C);
TOWER(n-1, B, A, C)
}

Algorithm analysis:
 Input size? Basic operation?
22
EXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
if n<1 return;
TOWER(n-1, A, C, B);
Move(A, C);
TOWER(n-1, B, A, C)
}

Algorithm analysis:
 Do we need to differentiate best case, worst
case & average case for inputs of size n?
23
EXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
if n<1 return;
TOWER(n-1, A, C, B);
Move(A, C);
TOWER(n-1, B, A, C)
}

Algorithm analysis:
 Set up a recurrence relation and initial
condition(s) for C(n)
24
EXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
if n<1 return;
TOWER(n-1, A, C, B);
Move(A, C);
TOWER(n-1, B, A, C)
}

Algorithm analysis:
 C(n) = 2C(n-1)+1
25
In-Class Exercise


P. 76 Problem 2.4.1 (c): solve this recurrence relation:
x(n) = x(n-1) + n for n>0, x(0)=0
P. 77 Problem 2.4.4: consider the following recursive
algorithm:
• Algorithm Q(n)
// Input: A positive integer n
If n = 1 return 1
else return Q(n-1) + 2 * n – 1
• A. Set up a recurrence relation for this function’s values and
solve it to determine what this algorithm computes
• B. Set up a recurrence relation for the number of
multiplications made by this algorithm and solve it.
• C. Set up a recurrence relation for the number of
additions/subtractions made by this algorithm and solve it.
26
Example: BinRec(n)
Algorithm BinRec(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary representation
if n = 1 return 1
else return BinRec( n/2 ) + 1
27
Smoothness rule
If T(n)  (f(n))
for values of n that are powers of b, where b  2,
then
T(n)  (f(n))
28
Example: BinRec(n)
Algorithm BinRec(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary representation
if n = 1 return 1
else return BinRec( n/2 ) + 1
If C(n)  (f(n))
for values of n that are powers of b, where b  2,
then
C(n)  (f(n))
29
Fibonacci numbers

The Fibonacci sequence:
0, 1, 1, 2, 3, 5, 8, 13, 21, …

Fibonacci recurrence:
F(n) = F(n-1) + F(n-2)
F(0) = 0
F(1) = 1
2nd order linear homogeneous
recurrence relation
with constant coefficients
30
Solving linear homogeneous recurrence
relations with constant coefficients

Easy first: 1st order LHRRCCs:
C(n) = a C(n -1)

C(0) = t
Extrapolate to 2nd order
L(n) = a L(n-1) + b L(n-2)




… Solution: C(n) = t an
… A solution?: L(n) = ?
Characteristic equation (quadratic)
Solve to obtain roots r1 and r2 (quadratic formula)
General solution to RR: linear combination of r1n and r2n
Particular solution: use initial conditions
31
Solving linear homogeneous recurrence
relations with constant coefficients

Easy first: 1st order LHRRCCs:
C(n) = a C(n -1)

C(0) = t
… Solution: C(n) = t an
Extrapolate to 2nd order
L(n) = a L(n-1) + b L(n-2)
… A solution?: L(n) = ?

Characteristic equation (quadratic)
Solve to obtain roots r1 and r2 (quadratic formula)
General solution to RR: linear combination of r1n and r2n
Particular solution: use initial conditions

Explicit Formula for Fibonacci Number: F(n) = F(n-1) +F(n-2)



32
Computing Fibonacci numbers
1. Definition based recursive algorithm
Algorithm F(n)
// Compute the nth Fibonacci number recursively
//Input: A nonnegative integer n
//Output: the nth Fibonacci number
if n  1 return n
else return F(n-1) + F(n-2)
33
Computing Fibonacci numbers
2. Nonrecursive brute-force algorithm
Algorithm Fib(n)
// Compute the nth Fibonacci number iteratively
//Input: A nonnegative integer n
//Output: the nth Fibonacci number
F[0]  0; F[1]  1
for i  2 to n do
F[i]  F[i-1] + F[i-2]
return F[n]
34
Computing Fibonacci numbers
3. Explicit formula algorithm
1 1 5 n
F (n) 
(
) rounded to the nearest integer
5
2
Special care in its implementation:
Intermediate results are irrational numbers
Their approximations in the computer are accurate enough
Final round-off yields a correct result
35
In-Class Exercises

What is the explicit formula for A(n)?
A(n) = 3A(n-1) – 2A(n-2)

A(0) = 1
A(1) = 3
P.83 2.5.3. Climbing stairs: Find the number of
different ways to climb an n-stair stair-case if each
step is either one or two stairs. (For example, a 3stair staircase can be climbed three ways: 1-1-1, 12, and 2-1.)
36