Download Chapter 6

CHAPTER 6
Mathematical Induction and Recursion
References - (Nothing in Brooks)
Mark Allen Weiss, Data Structures and Algorithm Analysis, The
Benjamin/Cummings Publishing Company, Redwood City, CA
Alan Tucker, Applied Combinatorics, John Wiley & Sons, New York
Mathematical Induction -- Two steps
1. Prove theorem is true for some small value (degenerate case), say a.
2. Assume theorem is true for some k  a . (or even all values between a and k).
Prove theorem is true for k+1.


If we can do that, then we know theorem is true for all values n  a .
That is, we can "get" to any value n by starting at a, and then stepping up 1
until we reach n.
k+2
k+1
k
4
3
2
1
Copyright (c) 1999 by Robert C. Carden IV, Ph.D.
6/22/2017
Mathematical induction and recursion
A simple induction proof
n
n n  1
.
Prove that for all n  1,  i  
2
i 1
example
PROOF
base case :
n 1
11  1
2
i 1
Assume true for n = k, i.e.
1
 i  1  1
k
i

i 1
2
2

k  k  1
2
Then
k 1
i


k  1
k  k  1

2
k  k  1  2k  1
2


k  1(k  2)

2
k  1
k
i
i 1
i 1



factor out k  1
6-2
Mathematical induction and recursion
A bad induction proof
We will prove that all elements x1 , , xn in a set S n are equal.
(1) Initial step (n = 1) : S1 has one element x1 equal to itself.
(2) Induction step : Assume x1  x2  xn 1.
Since xn 1  xn by the induction assumption , then x1  x2  xn 1  xn .
Another bad induction proof
We will now prove that for a  0, a n  1.
(1) Initial step (n = 0) : a 0  1 : This is always true.
(2) Induction step : Assume a n-1  1.
a n 1a n 1 1 1
Thus, a 

 1.
n2
a
1
n
6-3
Mathematical induction and recursion
Proving a property about Fibonacci numbers
example
We wish to prove that the Fibonacci numbers,
i0
1


Fi  
1
F  F
i 2
 i 1
satisfy th e property t hat
i 1
i2
i
5
Fi    , for i  1.
 3
Note that this property is not true for i  0.
basis
Here we must verif y that thi s is true for both F1 and F2 because
the induction step will look back two steps (i.e. to deduce F3
needs F1 and F2 in order to perform the computatio n)
case i = 1 :
F1  1 
5
3
2
25
18 25
5
case i = 2 :
F2  F1  F0  2 and    . Thus, 2  
9
9
9
 3
Assume true for some k  2 such that it is true for i  1,2,  , k .
5
We need to prove that Fk 1   
 3
k 1
k
Thus Fk 1
 5  5
    
 3  3
 Fk  Fk 1
 3  5 
   
 5  3 
k 1
 15  9  5 
=
 
 25  3 
5
Therefore, Fk 1   
 3
2
 3  5
   
 5  3
k 1
k 1
24  5 
  
25  3 
k 1
 3 9  5 
    
 5 25  3 
k 1
k 1
6-4
 5
 
 3
k 1
k 1
Mathematical induction and recursion
Proof that any integer n>1 can be written as a product of primes
A prime number is an integer p>1 that is divisible by no other integer besides 1
and p
Theorem
Any integer n>1 can be written as a product of prime numbers.
Proof (by induction)
The initial step is to prove that 2 can be written as a product of primes. But 2 is
itself prime. Thus 2 is trivially the product of primes, i.e. it is itself a prime.
Now assume that the numbers 2, 3,...,n-1 can be written as a product of primes.
We will use this assumption to prove that n can also be written as a product of
primes.
If n is prime, then it can trivially be written as a product of primes. Suppose n is
not prime. Then, there exists an integer m which divides n.
If k=n/m, then n=k m. Since k and m must be less than n and greater than 1, they
too can each be written as a product of primes. By multiplying these two prime
products for k and m, one obtains the desired representation of n as a product of
primes.
6-5
Mathematical induction and recursion
Recursion -- introduction
Reference: Brooks, Chapter 5 (5.4)

Consider the definition of n! (i.e. n factorial)

One way to look at it:
n! 1 2  3  n
0! 1
n! n  n  1!

Another way to look at it:

To compute 4! using the second definition, we would proceed as follows:
0! 1
4!  4  3!
 4  6  24
3  2!
2  1!
 3 2 
6

2 1 
2
1 1

1

1
1  0! 
1

We employ the procedure of computing n! recursively until we reach the base
case of its definition
6-6
Mathematical induction and recursion
Recursion -- introduction (2)




Our description of n! is an example of a recurrence relation
A recurrence relation is simply a function which is defined recursively
For example, the following gives a recurrence relation for n!
n  f n  1
f n   
1

if n > 0
if n = 0
To prove that f(n) = n!, we can use mathematical induction
example
Prove that f(n) = n!
base case :
n0
f 0   1  0!
Assume f (k ) = k! , for some k  0
We need to prove that f k  1  k  1!
f k  1

k  1  f k 

(k  1)  k!

(k  1)!
Therefore, for all n  0, f n   n!
6-7
Mathematical induction and recursion
Recursion -- introduction (3)

We can implement a function in C to compute n! using either iteration or
recursion
n  f n  1
f n   
1

if n > 0
if n = 0
Iterative C implementation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
long factorial (long n)
{
long i, result = 1;
if (n < 0)
return 0;
if (n == 0)
return 1;
for (i = 1; I <= n; i++)
result *= i;
return result;
}
Recursive C implementation
1
2
3
4
5
6
7
8
9
long factorial (long n)
{
if (n < 0)
return 0;
else if (n == 0)
return 1;
else
return n * factorial (n - 1);
}
6-8
Mathematical induction and recursion
Recursion -- introduction (4)



Each block is given its own activation record containing all automatic variables
Consider calling the recursive factorial(4)
Let us see what the activation frame looks like at each level
1
2
3
4
5
6
7
8
9
long factorial (long n)
{
if (n < 0)
return 0;
else if (n == 0)
return 1;
else
return n * factorial (n - 1);
}
Initial call to factorial(4)
factorial(4)
n
return
4
4*factorial(3)
6-9
Mathematical induction and recursion
Recursive call to factorial(3)
1
2
3
4
5
6
7
8
9
long factorial (long n)
{
if (n < 0)
return 0;
else if (n == 0)
return 1;
else
return n * factorial (n - 1);
}
n
factorial(3)
return
n
factorial(4)
3
return
3*factorial(2)
4
4*factorial(3)
6-10
Mathematical induction and recursion
Recursive call to factorial(2)
1
2
3
4
5
6
7
8
9
long factorial (long n)
{
if (n < 0)
return 0;
else if (n == 0)
return 1;
else
return n * factorial (n - 1);
}
n
factorial(2)
return
n
factorial(3)
2*factorial(1)
3
return
n
factorial(4)
2
return
3*factorial(2)
4
4*factorial(3)
6-11
Mathematical induction and recursion
Recursive call to factorial(1)
1
2
3
4
5
6
7
8
9
long factorial (long n)
{
if (n < 0)
return 0;
else if (n == 0)
return 1;
else
return n * factorial (n - 1);
}
n
factorial(1)
return
n
factorial(2)
2
2*factorial(1)
3
return
n
factorial(4)
1*factorial(0)
return
n
factorial(3)
1
return
3*factorial(2)
4
4*factorial(3)
6-12
Mathematical induction and recursion
Recursive call to factorial(0)
1
2
3
4
5
6
7
8
9
long factorial (long n)
{
if (n < 0)
return 0;
else if (n == 0)
return 1;
else
return n * factorial (n - 1);
}
n
factorial(0)
return
n
factorial(1)
1*factorial(0)
2
2*factorial(1)
3
return
n
factorial(4)
1
return
n
factorial(3)
1
return
n
factorial(2)
0
return
3*factorial(2)
4
4*factorial(3)
6-13
Mathematical induction and recursion
Returning from the recursive call to factorial(0)
1
2
3
4
5
6
7
8
9
long factorial (long n)
{
if (n < 0)
return 0;
else if (n == 0)
return 1;
else
return n * factorial (n - 1);
}
n
factorial(1)
return
n
factorial(2)
return
1*1
2
2*factorial(1)
3
return
n
factorial(4)
1
return
n
factorial(3)
1
3*factorial(2)
4
4*factorial(3)
6-14
Mathematical induction and recursion
Returning from the recursive call to factorial(1)
1
2
3
4
5
6
7
8
9
long factorial (long n)
{
if (n < 0)
return 0;
else if (n == 0)
return 1;
else
return n * factorial (n - 1);
}
n
factorial(2)
return
n
factorial(3)
2
return
2*1
3
return
n
factorial(4)
2
3*factorial(2)
4
4*factorial(3)
6-15
Mathematical induction and recursion
Returning from the recursive call to factorial(2)
1
2
3
4
5
6
7
8
9
long factorial (long n)
{
if (n < 0)
return 0;
else if (n == 0)
return 1;
else
return n * factorial (n - 1);
}
n
factorial(3)
return
n
factorial(4)
return
3
6
3*2
4
4*factorial(3)
6-16
Mathematical induction and recursion
Returning from the recursive call to factorial(3)
1
2
3
4
5
6
7
8
9
long factorial (long n)
{
if (n < 0)
return 0;
else if (n == 0)
return 1;
else
return n * factorial (n - 1);
}
n
factorial(4)
return
4
24 4*6
6-17
Mathematical induction and recursion
Recursion -- how it works









Recall that each block is given its own activation record containing all
automatic variables
The recursive implementation of factorial uses the system stack to store
intermediate values
On the other hand, the iterative implementation of factorial uses only one
variable to store the intermediate results
Thus, we observe that the amount of space required by the iterative version is
constant
The recursive version, though, requires a unique activation frame for each
function call
The number of activation frames (at the worst case) is n+1
Thus, the space requirement of the recursive version is linear in the number of
recursive calls!
To wit: the iterative version (in this case) uses a lot less system resources than
the recursive version
 This is because the best algorithm for computing n! requires constant space
because we only need to know the previous value
Recursive implementations are appropriate where the amount of space
consumed would have been consumed anyway
 i.e., we would have had to stack the input anyway
6-18
Mathematical induction and recursion
Recursion -- concepts and program design
Simple mathematical formulas
5
C  ( F  32)
9
(Farenheit  Celcius)
Recursive formulas
x0
0

f x   
2
2 f  x  1  x
x0
C implementation
1
2
3
4
5
6
7





long f (long x)
{
if (x == 0)
/* base case */
return 0;
else
return 2 * f (x - 1) + x * x;
}
Lines 3 and 4 handle the base case, i.e. the value which is known without
resorting to recursion
Line 6 makes the recursive call
Without lines 3 and 4, f would go into an infinite loop
Note that an attempt to evaluate f(-1) will result in calls to f(-2), f(-3), and so
on
Because this never gets to a base case, the program will never terminate with
any of those values
6-19
Mathematical induction and recursion
Recursion (2)

The following routine is a bad use of recursion because it never terminates
1
2
3
4
5
6
7





long bad (long n)
{
if (n == 0)
return 0;
else
return bad (n / 3 + 1) + n - 1;
}
Calling bad(1) will call bad(1) at line 6 and go into an infinite loop
Calling bad(2) will call bad(1) which will once again go into an infinite loop
In addition, bad(3), bad(4), and bad(5) all call bad(2) which goes into an
infinite loop
The only value of n for which this program works is its special case, i.e. n=0
These examples illustrate the necessity for the first two fundamental rules of
recursion
1. Base cases

You must always have some base case which can be solved without recursion
2. Making progress

For the cases that are to be solved recursively, the recursive call must always be
to a case that makes progress toward a base case
6-20
Mathematical induction and recursion
Recursion (3) -- printing out numbers

Suppose we wish to write a program in a language to print out a positive
integer n but where the only I/O routines available will take a single digit
number and print it to the terminal
Given:
Implement:
void print_digit (int digit);
void print_number (int n);
Solution (recursive)



To print out the number 67234, we need to first print out 6723 followed by a 4
We accomplish the second step by calling
print_digit (n % 10);
We can achieve the first part by calling
print_number (n/10); /* integer division */
1
2
3
4
5
6
7
8
9
10
11
12
int print_number (int n) {
if (n < 0) {
/* error -- n is negative */
return –1;
} else if (n < 10) {
print_digit (n);
} else {
print_number (n / 10); /* recursive call */
print_digit (n % 10);
}
return 0;
}
6-21
Mathematical induction and recursion
Recursion (4) -- printing out numbers
1
2
3
4
5
6
7
8
9
10
11
12
int print_number (int n) {
if (n < 0) {
/* error -- n is negative */
return -1;
} else if (n < 10) {
print_digit (n);
} else {
print_number (n / 10);
print_digit (n % 10);
}
return 0;
}
Theorem
The recursive number printing algorithm is correct for n  0.
Proof (by induction)
Base case
If n has one digit, 0  n  9 and print_digi t is called directly.
Thus, the algorithm is trivially correct.
Assume
algorithm works for k - digit number.
We want to prove that it works for a k + 1 digit number.
A k + 1 digit number is expressed by its first k digits followed by its
least significa nt digit.
But the number formed by the first k digits is exactly n 10 , which by our
hypothesis is correctly printed out.
The last digit is exactly n%10 which is also correctly printed, immediatel y
after the first k digits.
Thus, the program prints out any k  1 digit number correctly.
6-22
Mathematical induction and recursion
Recursion (5)





Notice that the proof is almost identical to the algorithm description.
This illustrates that when designing a recursive program, all smaller instances
of a problem may be assumed to work correctly.
The recursive program combines solutions to the smaller problems which had
to be solved immediately.
The mathematical justification is proof by induction.
This is a very profound concept
 Think about the justification behind mathematical induction
 Think about what a computer does when it encounters a recursive call and
when the actual work is done
 A lot of bookkeeping is done on the stack...
Four fundamental rules when writing a recursive program
1. Base cases

You must always have some base case which can be solved without recursion
2. Making progress

For the cases that are to be solved recursively, the recursive call must always be
to a case that makes progress toward a base case
3. Design rule

Assume that all recursive calls works (like in mathematical induction)
4. Compound interest rule

Never duplicate work by solving the same instance of a problem in separate
recursive calls
6-23
Mathematical induction and recursion
Analyzing recursive procedures


The total time for a recursive procedure is usually a recurrence relation
In some cases, the recursion is a thinly veiled for loop; in this case, analyze it
as if it were a for loop
example
1
2
3
4
5
6
7





int factorial (int n)
{
if (n == 0)
return 1;
else
return n * factorial (n - 1);
}
By inspection, we can see that factorial is O(n)
However, to prove it, we would need to derive a recurrence relation and then
prove an upper bound on that recurrence
Due to the bookkeeping costs required by recursion, it also uses O(n) space (on
the stack)
A loop version can be written using constant space
Recursion is appropriate when you implicitly would want to use a stack to store
intermediate results
 We will see that tree traversal is one good example
 Recursive descent parsing is another good example
6-24
Mathematical induction and recursion
A formal derivation of the time bound for factorial
1
2
3
4
5
6
7
int factorial (int n)
{
if (n == 0)
return 1;
else
return n * factorial (n - 1);
}
If n = 0, the running time is that of line 3 + line 4 which is O1
If n  0, the running time is that of line 3 + line 6
The running time of line 6 is T n  1  1
Therefore, the total time for this function is
n0
1

T n   
T n  1  2
n0
Claim T n   2n  1
Proof (by induction)
basis
n  0 : T 0   1  (2  0)  1
Assume true for n  k , i.e. assume T k   2k  1.
Then,
T k  1



T k   2
2k  1  2
2k  1  1
Therefore, T n   2n  1,n  0.
Because T n   2n  1, we can conclude that factorial is On .
6-25
Mathematical induction and recursion
An example of how not to use recursion
1
2
3
4
5
6
7








int fib (int n)
{
if (n <= 1)
return 1;
/* 0 or 1 */
else
return fib (n - 1) + fib (n - 2);
}
To analyze this program, let T(n) = running time for fib(n)
If n is either 0 or 1, the running time is that of lines 3 and 4 which is constant,
i.e. O(1)
If n > 2, the time is equal to sum of
 the time it takes to execute line 3
 the time it takes to do line 6
Line 6 consists of two function calls plus an addition
Thus, line 6 requires T(n-1) + T(n-2) + 1
Consequently, for n  2, adding lines 3 and 6 yields
T(n)
= T(n-1) + T(n-2) + 1 + 1
= T(n-1) + T(n-2) + 2
We can prove by induction that T n  fibn
 5


fib
n

 
We can also prove by induction that
 3
 3
fibn    
 2
n
n

In addition, we can prove that

 3
Thus, T n     , which implies that the running time of function fib grows
 2
n

exponentially
This is very bad
6-26
Mathematical induction and recursion
The reason why function fib is so bad
1
2
3
4
5
6
7




int fib (int n)
{
if (n <= 1)
return 1;
/* 0 or 1 */
else
return fib (n - 1) + fib (n - 2);
}
There is a huge amount of redundant work being done
This violates the fourth major rule of recursion
 the compound interest rule
The first call on line 6, fib(n-1), computes fib(n-2) at some point and then
throws it away
This value is then recomputed by the second call to fib, i.e. fib(n-2)
Basic maxim
DO NOT COMPUTE
ANYTHING MORE
THAN ONCE
6-27
Mathematical induction and recursion
The much faster iterative version of fib
1 long fib (long n)
2 {
3
long i, back_1, back_2, result;
4
5
back_2 = 1;
6
back_1 = 1;
7
if (n <= 1)
8
return 1;
/* 0 or 1 */
9
for (i = 2; i <= n; i++) {
10
result = back_1 + back_2;
11
back_2 = back_1;
12
back_1 = result;
13
}
14
return result;
15 }


In this example, we observe that we only need to know the last two computed
values in order to compute the next value
This follows directly from the recurrence relation
i0
1


Fi  
1
F  F
 i 1 i  2



i 1
i2
The loop computes each Fibonacci number by starting at 2 and working its way
upward
Clearly, the number of iterations is bounded above by n
The amount of space required is constant
6-28
Mathematical induction and recursion
A good example of how to use recursion -- binary search

The binary search problem is given as follows.
Given an integer x and integers a0 , a1 , a2 ,, an , which are presorted and already in
memory, find i such that ai  x, or return i  1 if x is not in the input.








One solution would be to scan through the input from left to right
This algorithm, though, obviously runs in linear time
It is far better to derive an algorithm which takes advantage of the property that
the input is sorted and already in memory
We want our program to search for the object the same way we search for a
word in a dictionary
The program will look in the middle of the input
If it finds the element, it will return the index
Otherwise, it will narrow its search to either the lower or upper half of the input
In this sense, it is able to cut its search in half, thus its name binary search
low
mid-1
mid+1
mid
6-29
high
Mathematical induction and recursion
The binary search function
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
/**
** To call this function, pass in the array
** a[], the lower bound (0) and the upper
** bound (n-1). This function returns the
** index of the key within a[].
**/
int binsearch (int a[], int low, int high, int key)
{
int mid = (low + high) / 2;
if (low > high) {
/* not in array */
return -1;
}
if (key < a[mid]) /* recursive call */
return binsearch (a, low, mid - 1, key);
else if (key > a[mid]) /* recursive call */
return binsearch (a, mid + 1, high, key);
else /* if (key == a[mid]) */
return mid;
/* found it */
}
low
mid-1
mid+1
high
mid




The complexity of this function equals the number of calls to this function
Because this function always divides its input in half before proceeding, the
number of calls to this procedure is bounded above by log(n)
Also, the amount of space used on the stack is also bounded by log(n)
It is clearly possible to write an equivalent iterative version which uses O(1)
space
 the iterative version is marginally better than the recursive version
 because log(n) tends to be very small even for large input, the difference in
efficiency is marginal indeed
6-30
Mathematical induction and recursion
Input string reversal -- an even better example of recursion



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
Consider the following two routines to reverse a line of input from the terminal
One works and the other doesn't
The one that works uses O(n) space (as it should)
#include <stdio.h>
void reverse (void)
{
int ch = getchar ();
switch (ch) {
case '\n':
putchar (ch);
case EOF:
break;
default:
reverse ();
putchar (ch);
}
}
void reverse2 (void)
{
static int ch = getchar ();
switch (ch) {
case '\n':
putchar (ch);
case EOF:
break;
default:
reverse2 ();
putchar (ch);
}
}
Beware of “non-reentrant” functions…
6-31
Mathematical induction and recursion
Precedence and order of evaluation
Operator
() [] -> .
!
~ ++ -(unary) + (indirection) *
&
( type )
sizeof
*
/
%
+
<<
>>
<= < > >=
==
!=
&
^
|
Associativity
left to right
Order of Evaluation
-
right to left
-
left to right
left to right
left to right
left to right
left to right
left to right
left to right
left to right
&&
left to right
||
left to right
?:
right to left
left to right
sequence point after
first argument
short circuit
left to right
sequence point after
first argument
short circuit
first operand eval
sequence point after
first argument
=
+= -=
*= /= %=
&= |=
<<= >>=
,
right to left
-
left to right
left to right
sequence point after
first argument
6-32
Mathematical induction and recursion
Parse trees
example:


x = a + b * c
We wish to draw the parse tree for this string
First, we must fully parenthesize it:
x = (a + (b * c))

Now we can draw the parse tree
=
x
+
*
a
c
b


The higher precedence operand is lower in the tree.
To evaluate the expression, do a preorder (or postorder) traversal of the tree.
preorder:
inorder:
postorder:
visit root, traverse left subtree, traverse right subtree
traverse left subtree, visit root, traverse right subtree
traverse left subtree, traverse right subtree, visit root
6-33
Mathematical induction and recursion
example -- continued
=
x
+
a
*
c
b
preorder:
inorder:
postorder:
visit root, traverse left subtree, traverse right subtree
traverse left subtree, visit root, traverse right subtree
traverse left subtree, traverse right subtree, visit root
prefix string:
infix string:
postfix string:
=
x
x
=
a
x
a
b
+
+
c
a
b
*
*
*
+
6-34
b
c
=
c
(like HP calculator)
Mathematical induction and recursion
Standard exercises
1. Given expression, draw parse tree
2. Given parse tree, derive infix expression with minimal parentheses
3. Given either prefix or postfix expressions, derive infix expression (Hint: draw
the parse tree)
example
*
+
-
-
e
-
*
a
-
b
c
d
f
i
h
g
infix string (without parentheses)
a * b - c - d + e * f - g - h - i
C expression with required parentheses
(a * b - (c - d) + e) * (f - g - h - i)
6-35
Mathematical induction and recursion
example
Give the parse tree for:
a << b + c * d & e | f & g ? h == i : j
answer
1. Fully parenthesize it
(((a << (b + (c * d))) & e) | (f & g)) ? (h == i) : j
2. Draw the parse tree
?:
|
==
&
&
<<
e
a
f
h
g
+
b
*
c
d
6-36
j
i