Download Mathematical induction - Department of Information Technology

Mathematical induction
If we want to prove that P n holds for for all natural numbers n, we can do the following twostep rocket called mathematical induction:
1. Prove that P 0 holds
2. Prove that if P n holds, then P n1 holds. Ie prove ∀ n∈ℕ: P n⇒ P n1
Step 1 is called the base case, step 2 is called the inductive step. In step 2, the assumption that
P n holds is called the inductive hypothesis.
Example
Prove that for all n∈ℕ , nn 2 is an even number.
Base case: prove that 002 is an even number
002 =0 , which is obviously even.
Inductive step: assuming that nn 2 is even, prove that n1n12 is even.
Since we know that nn 2 is even, we define nn 2=2m . This yields:
n1 n12
=n1 n22n1
=n2 n2n2
=2m2n2
=2 mn1
...which is obviously an even number, since it's a multiple of two.
2009 Johannes Åman Pohjola
1/6
Dept. of Information Technology
A slight generalization
If we want to prove P n for all n≥n 0 , we can use P n 0 rather than P n as the base case.
Example
n
n2
Show that for all n≥12 , ∑ i−3≥
4
i=0
12
Base case: show that
∑ i−3≥
i=0
122
4
12
∑ i −3
i =0
=−3−2−1−0123456789
=39
≤36
12 2
=
4
n
Inductive step: show that if
∑ i−3≥
i=0
n2
and n≥12 , then
4
n1
∑ i−3≥
i=0
 n12
4
n1
∑ i −3
i=0
n
=∑ i−3n1−3
i =0
2
n
n−2
4
n24n−8
=
n−2
4
n22n12n−9
=
4
2
n 2n12n−9
=
4
2
n1 2n−9
=
4
2
n1
≥
4
≥
2009 Johannes Åman Pohjola
2/6
Dept. of Information Technology
A larger generalization: structural induction
The set ℕ of natural numbers can be inductively defined as follows:
•
0 is in a natural number
•
If n is a natural number, then n+1 is a natural number.
Notice the similarity in structure between this style of definition and inductive proofs. It's no
coincidence. Induction proofs exploit the property that the natural numbers are an inductivelydefined structure.
However, the natural numbers is far from the only structure which can be defined as such: sky's the
limit! Linked lists and binary trees are examples of well-known data structures that can be
inductively defined (the definitions are left as an exercise for the reader).
Clearly, if other structures can be inductively defined just as the natural numbers, then we should be
able to prove properties about those structures with induction, like we do for the natural numbers.
Example
Simple arithmetic expressions, involving only parentheses and binary operators, can be inductively
defined as follows:
•
All numbers and variables are arithmetic expressions
•
If A1 and A2 are arithmetic expressions, then so is
and A1 / A2 .
•
If
A 1 A 2 ,
A1 – A 2 ,
A1∗A2 ,
A is an arithmetic expression, then so is  A .
Numbers, variables, binary operators and parentheses are symbols. Let ∣A∣ denote the number of
symbols in an arithmetic expression A , ie ∣1 x∣=3 and ∣135∗7∣=7 .
Theorem: for all A, lline A rline is odd.
We prove this by induction. Since there are several (as opposed to just one) ”inductive steps” in the
definition of our structure, the proof will follow the same structure.
Base case(s): a number or variable has an odd number of symbols
A variable or number is by definition a symbol: we have one symbol, and one is odd.
Inductive step (i): if ∣A1∣ and ∣A2∣ are odd, then so is ∣A1 A2∣ .
We let ∣A1∣=2n1 and ∣A2∣=2m1 . Thus, we have:
∣A1 A2∣=∣A1∣∣A2∣1=2n12m11=2mn11
Inductive step (ii): if ∣A1∣ and ∣A2∣ are odd, then so is ∣A1− A2∣ .
Proof analogous to (i).
…
Inductive step (v): if ∣A∣ is odd, then so is ∣ A∣ .
Let ∣A∣=2n1 . We have ∣ A∣=2∣A∣=22n1=2 n11 .
2009 Johannes Åman Pohjola
3/6
Dept. of Information Technology
Cargo cult induction
We wish to show that all horses have the same colour, or equivalently, that for every set of horses,
all its members have the same colour. We use induction over the size of the horse sets.
Base case: all horses in the empty set have the same colour.
Vacuously true.
Inductive step: if all horses in horse sets of size n have the same colour, then so do all horses in
sets of size n+1.
Let H ={h1, h2, … , hn , hn1 } denote any size n+1 set of horses. Now consider the sets
H 1={h2, h3, … , hn , hn1 } and H 2={h 1, h2, … , hn −1 , hn } . Since H 1 and H 2 each
have n elements, then by the induction hypothesis, we know that all horses in H 1 and
H 2 have the same colour. Since H 1 and H 2 overlaps, all horses in H must
have the same colour.
This completes our ”proof”. However, spider-sense would indicate that something is not right here:
the result we just obtained is widely held to be false. In fact, we can provide a counterexample:
Figure 1.1: Counterexample
Clearly, there is some error in our proof. Can you find it?
2009 Johannes Åman Pohjola
4/6
Dept. of Information Technology
Asymptotic notation: an example
We wish to prove that 5n 2−3n−6=n 2 . Unfolding the definition of  -notation, our task is to
find positive constants c 1 , c 2 and n 0 such that for all n≥n 0 we have
2
2
0≤c 1⋅n ≤5n −3n−6≤c 2⋅n
2
(*)
One general strategy is the following two-step rocket:
•
Guess values of c 1 , c 2 and n 0 . You can arrive at your guesses with more or less
sophistication: plot graphs, fiddle around algebraically or apply astrology according to
taste. It doesn't matter, as long as you're convinced that your guess is correct.
•
2
Prove that the values you guessed actually satisfy (*) - ie, prove 0≤c 1⋅n ,
c 1⋅n 2≤5n 2−3n−6 and 5n 2−3n−6≤c 2⋅n2 for all n≥n 0 .
Note that it doesn't matter which values you pick for the constants, as long as they satisfy the
inequality. Your task is to show that such constants exists, not to find the ”best” such constants.
Hence, the best choice is the most convenient choice for you: when in doubt, go for the constants
which simplify your proof the most.
In this example, I use forbidden arcane magical lore to guess c 1=1 , c 2=5 and n 0=2 . I then need
to prove:
•
2
0≤1⋅n for all n≥2
•
2
2
1⋅n ≤5n −3n−6 for all n≥2
•
2
2
5n −3n−6≤5⋅n for all n≥2
Proving the above inequalities is straightforward – feel free to do it as an exercise.
2009 Johannes Åman Pohjola
5/6
Dept. of Information Technology
Methods for solving recurrence equations
Recursion trees
•
Each node represents the cost of one subproblem, all the way down until the leafs, which
represent the cost of the base cases.
•
The sum of all node costs within a level gives the total cost of that level.
•
The sum of all per-level costs gives the total cost.
Substition
•
Guess a solution.
•
Verify the correctness of the solution using induction.
Example: merge sort
The following recurrence describes the runtime of merge sort, where n denotes the number of
elements we are sorting. Constants and lower-order terms have been somewhat simplified: we want
to illustrate an idea here, not do something incredibly hairy.
T 1=1
n
T  n=2⋅T ⌊ ⌋c⋅n
2
If you are at the tutorial session right now, you should be witnessing a nice recursion tree being
drawn on the blackboard! Basically, since there are Olg n levels, with the cost of each level
being O n , the cost of the entire tree is O n⋅lg n !
We can also show this with the substitution method. We begin by guessing that T  n≤c⋅n⋅lg nn
(which is what the recursion tree would indicate), and go with induction from there:
Base case: T 1≤c⋅1⋅lg 11
Simplifies to 1≤1 .
n
n
n
n
Inductive hypothesis: if T ⌊ ⌋≤c⋅⌊ ⌋⋅lg ⌊ ⌋⌊ ⌋ , then T  n≤c⋅n⋅lg nn
2
2
2
2
T n
n
=2⋅T ⌊ ⌋c⋅n
2
n
n
n
≤2⋅c⋅⌊ ⌋⋅lg ⌊ ⌋⌊ ⌋c⋅n
2
2
2
n
n n
≤2⋅c⋅ ⋅lg   c⋅n
2
2 2
=c⋅n⋅lg n−1nc⋅n
=c⋅n⋅lg nn
Notice how we use the fact that we're working with inequalities to our advantage. From here, it is
trivial to show that T  n=O n⋅lg n .
2009 Johannes Åman Pohjola
6/6
Dept. of Information Technology