Download Document

Induction
• Main objective: Using induction or inductive
reasoning as a “recursive” algorithm design
technique
• Why recursion
– Concise algorithms for complex problems can be
developed
– The algorithms are easy to comprehend
– Development time
– Proof of correctness of the designed algorithm is
usually simple.
How Does Induction Produce
Recursive Algorithms?
• Having a problem with input size n, it is
sometimes easier to start with a solution to
the problem with a smaller size and extend
the solution to include the input size n.
Problems to Discuss
•
•
•
•
•
•
Selection sort
Insertion sort
Radix sort
Integer exponentiation
Evaluating polynomials
Finding the majority element
Recursive Selection Sort
• Induction Hypothesis: We know how to sort
A[2..n]
• Inductive Reasoning: We sort A[1..n] as
follows
– Find the minimum A[j], 1  j  n
– Swap(A[1],A[j])
– Sort A[2..n] // induction hypothesis
• What is the base case?
• What is the initial call to the sorting algorithm?
• This kind of recursion is called tail-recursion.
Recursive Selection Sort Algorithm
Algorithm SELECTIONSORTREC
Input: An array A[1..n] of n elements.
Output: A[1..n] sorted in nondecreasing order.
Procedure sort(i) {Sort A[i..n]}
1. if i < n then
Begin
2.
k := i;
3.
for j := i + 1 to n
4.
if A[j] < A[k] then k := j;
5.
if k != i then Swap(A[i], A[k]);
6.
sort(i + 1);
End
Complexity Analysis of
Recursive Selection Sort
• What is the recurrence relation that
describes the number of comparisons
carried out by the algorithm?
• What is the solution to the recurrence?
Recursive Insertion Sort
• Induction Hypothesis: We know how to sort
A[1..n-1]
• Inductive Reasoning: We sort A[1..n] as
follows:
– Sort A[1..n-1] // induction hypothesis
– Insert A[n] in its proper position in A[1..n-1]
• This may involve copying zero or more elements
one position ahead in order to insert A[n]
Recursive Insertion Sort Algorithm
Algorithm INSERTIONSORTREC
Input: An array A[1..n] of n elements.
Output: A[1..n] sorted in non-decreasing order.
Procedure sort(i)
{Sort A[1..i]}
1. if i >1 then
2.
sort(i – 1)
{Recursive Call}
3.
x := A[i]
4.
j := i – 1
5.
while j > 0 and A[j] > x
6.
A[j + 1] := A[j]
7.
j := j – 1
end while
8.
A[j + 1] := x
end if
Complexity Analysis of
Recursive Insertion Sort
• Unlike selection sort, the analysis has to
differentiate between the best case and the
worst case. Why?
• Recurrence of the best case:
• Solution to the best case:
• Recurrence of the worst case:
• Solution to the worst case:
Radix Sort
• Treats keys as numbers in a particular radix
or base.
• This is NOT an elements comparison-based
sorting algorithm
• It only works under the following
assumption:
Radix Sort Derivation
• Assume that our keys are of the form
dk dk – 1 … d1
• Induction Hypothesis: Suppose that we know how to sort
numbers lexicographically according to their least k – 1
digits, dk – 1, dk – 2, …, d1 , k > 1.
• Inductive reasoning: We sort the numbers based on their
first k digits as follows:
– Use induction hypothesis to sort the numbers based on their
1..k-1 digits.
– Sort the numbers based on their corresponding kth digits
Description of Radix Sort Algorithm
1. Distribute the input numbers into 10 sublists L0,
L1,…, L9 according to the least significant digit, d1.
2. Form a new list, we denote by main list, by
removing the input from the 10 lists, starting from
L0, L1, …etc. in order.
3. Distribute the main list into the 10 sublists according
to the second digit, d2, and then repeat step 2.
4. Repeat step 3 for the rest of the digits in order of d3,
d4, …, dk.
Example
• Sort the following numbers using radix sort:
467 1247 3275 6792 9187
9134 4675 39 7
6644
Radix Sort Algorithm
Algorithm RADIXSORT
Input: A linked list of numbers L = {a1, a2,…,an} and k, the maximum number of
digits.
Output: L sorted in nondecreasing order.
1. for j ←1 to k do
2. Prepare 10 empty lists L0, L1, …, L9.
3. while L is not empty
4.
a ← next element in L. Delete a from L.
5.
i ← jth digit in a. Append a to list Li.
6. end while
7. L ← L0
8.
for i ←1 to 9 do
9.
L ← L . Li {append list Li to L}
10. end for
11. end for
12. return L
Time and Space Complexity Analysis
• Time Complexity
• Space Complexity
Integer Exponentiation
• What is the straightforward algorithm to
raise x to the power n?
• Such an algorithm is exponential in input
size!!!!!!!!
Algorithm Derivation
• Assume we have integer n which is represented in
binary as (bkbk-1…b1b0)2
• Induction Hypothesis: Assume that we know how
to compute xn/2.
• Inductive Reasoning: We raise x to the power n as
follows:
– Use induction hypothesis to compute
y=xn/2 . xn/2
– If n is even, xn = y
– If n is odd, xn = x . y
Integer Exponentiation Algorithm
ALGORITHM EXPRECURSIVE
Input: A real number x and a nonnegative integer n.
Output: xn. /* the first call to the algorithm is power(x,n) */
Procedure power(x, m) {Compute xm }
1. if m=0 then y ← 1
2. else
3.
y ← power(x,m/2)
4.
y ← y2
5.
if m is odd then y ← x . y
end if
7. return y
Complexity Analysis of the Algorithm
• When n = 0, the number of multiplications
is
Therefore, T(0)=
• Best Case Analysis:
– When does the best case occur?
– What is the recurrence equation? Solution?
• Worst Case Analysis:
– When does the worst case occur?
– What is the recurrence equation? Solution?
Iterative Exponentiation Algorithm
• Assume we have integer n which is represented
in binary as (bkbk-1…b1b0)2
 Starting with y = 1, scan the binary digits of the
number n from left to right (j = k down to 0):
– If bj = 0, square y
– If bj = 1, square y and multiply it by x
• Example: Compute 212 using the iterative
exponentiation algorithm
Polynomial Evaluation
• A polynomial of degree n is generally written as:
Pn ( x)  an x  an 1 x
n
n 1
 ...  a1 x  a0
where, a0, a1, …, an and x, is a sequence of n + 2
real numbers. Our objective here is to evaluate
this general polynomial at the point x.
Polynomial Representation
• Dense Representation
(n, a n , a n 1 ,..., a1 , a 0 )
– Disadvantage:
• Sparse Representation
( n, a n , i1 , a i1 , i 2 , a i2 ,..., i k , a ik )
– Disadvantage:
• Examples:
Two Straightforward Solutions
1. Evaluate each term aixi separately
•
What is its time complexity?
2. Compute xi by multiplying x by the previously
computed value xi-1
1. P  a 0 ;
2. t  1;
3. for i  1 to n do
t t*x
4.
P  P  ai * t
5.
•
How many multiplications, assignments, and
additions do we have?
Horner’s Rule
Pn ( x)  an x  an 1 x
n
n 1
 ...  a1 x  a0
 ((...((( an x  an 1 ) x  an  2 ) x  an 3 )...) x  a1 ) x  a0
 xPn 1 ( x)  a0
Horner’s Rule
• Induction Hypothesis: Suppose we know how to
evaluate the following:
Pn1 ( x)  an x
n 1
 an1 x
n2
 ...  a2 x  a1
• Inductive Reasoning: We evaluate Pn(x) as
follows:
– Using induction hypothesis, we know how to
evaluate Pn-1(x)
– Pn(x) = x Pn-1(x) + a0
• What is the base step?
Horner’s Algorithm
• Horner’s Algorithm
Input: A sequence of n+2 real numbers a0, a1, …, an, x
Output: Pn(x)=anxn+an-1xn-1+…+a1x+a0
p ← an
for j ← 1 to n do
p ← x . p + an-j
return p
• How many multiplications, additions, and assignments do
we have?
Finding the Majority Element
• Definition: Given a sequence of n integers, say
A[1..n], an element in A is called the majority
element if it appears more than n 2 times in A.
• Objective: To use induction to develop an
efficient algorithm that finds the majority
element in a sequence of n integers, if such
element exists.
– In any sequence of n elements, how many majority
elements can there be?
Examples
• What is the majority element in the
following examples:




1,6,8,3,4,1,6,8,1,6,1,1
1,6,8,3,4,1,6,1,6,1,1
1,6,3,4,1,6,1,6,1,1
1,6,4,1,6,1,6,1,1
Various Algorithms to Find the Majority
• The brute-force method:
– Its cost:
• Compare elements from 1 to n / 2 with every
other element.
– Its cost:
• Sort the elements and count how many times each
element appears in the sequence.
– Its cost
Using Induction to Find the Majority
• Theorem: If two different elements in the
original sequence are removed, then the
majority element in the original sequence
remains the majority element in the new
sequence.
Proof:
Procedure Candidate
• This procedure applies the previous theorem by
“eliminating” pairs of distinct elements and returning a
possible majority element:
Procedure candidate (m)
1. j  m; c  A[m]; count  1
2. while j < n and count > 0
3.
j  j + 1
4.
if A[j]=c then count  count + 1
5.
else count  count – 1
6. end while
7. if j = n then return c
8. else return candidate(j+1)
Finding the Majority
• The idea of algorithm Majority is as follows:
– Find a candidate majority element using the
procedure candidate
– Check whether this candidate element is a
majority element or not.
Algorithm Majority
Algorithm MAJORITY
Input: An array A[1..n] of n elements.
Output: The majority element if it
exists; otherwise none.
1. c  candidate(1)
2. count  0
3. for j  1 to n
4.
if A[j]=c then
count  count + 1
5. end for
6. if count > n/2 then return c
7. else return none
Example
Time Complexity Analysis of
Algorithm Majority
• The time complexity is calculated in terms of the
number of element comparisons.
• How many element comparisons do we have in
Procedure candidate? How many element
comparisons do we have in Algorithm Majority
(excluding those performed within Procedure
candidate)
• Hence, Algorithm Majority will perform
…………….. Comparisons in total.
• Which of all the algorithms discussed, related to
finding the majority element, is better?