Download Divide and Conquer

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.
CS 312: Algorithm Design &
Analysis
Lecture #6: Divide and Conquer
Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick
Announcements
 Proj. #1 due Today
 Monday: MLK Holiday
 Screencast Lecture #7 & Quiz #2
 Due by end of day Tuesday
 HW #4 due Wednesday at beginning of class
 We’ll work on HW #5 Wednesday in class together
 The other half of the Lecture #7 “flip”
Objectives
 Introduce Divide and Conquer
 Apply to Multiplication
 Analyze by defining a recurrence relation
 Introduce the Convex Hull problem, time
permitting
Thought
“Nothing is particularly hard if you divide it
into small jobs.”
-- Henry Ford
...
...
...
Divide and Conquer
Solve the smaller pieces.
Been there
 Which divide and conquer algorithms have
we already encountered?
Recall Efficiency of Multiplication
Algorithms
 Problem: Multiplication
 Algorithms:





American
British
a la Francaise / Russe
Arabic
…
 Efficiency so far:
Divide and Conquer
0502
5001
05
05
02
02




50
01
50
01
Is it correct?
shift
shift
shift
shift
4
2
2
0
= 2500000
=
500
=
10000
=
2
2510502
How long does it take on an input of size n?
5001
x 502
10002
0
2500500
2510502
𝑂(𝑛2)
Why isn’t it faster?
Can we do better?
 Yes.
 We shall see how …
Divide and Conquer
w x
0502
5001
y z
05
05
02
02
x
x
x
x
50
01
50
01
shift
shift
shift
shift
4
2
2
0
= 2500000
=
500
=
10000
=
2
2510502
product = wy104 + (wz + xy)102 + xz
wy104 +
wz102 +
xy102 +
xz100
Divide and Conquer
w x
0502
5001
y z
05
05
02
02
x
x
x
x
50
01
50
01
shift
shift
shift
shift
4
2
2
0
= 2500000
=
500
=
10000
=
2
2510502
product = wy104 + (wz + xy)102 + xz
wy104 +
wz102 +
xy102 +
xz100
wx
yz
Divide and Conquer
product = wy104 + (wz + xy)102 + xz
Let r = (w + x)(y + z) = wy + wz + xy + xz
Then wz + xy = r – wy - xz
product = wy104 + (r – wy - xz)102 + xz
= wy104 + ((w+x)(y+z)– wy - xz)102 + xz
From four sub-instances to three!
wx
yz
Divide and Conquer
product = wy104 + (wz + xy)102 + xz
Let r = (w + x)(y + z) = wy + wz + xy + xz
Then wz + xy = r – wy - xz
product = wy104 + (r – wy - xz)102 + xz
= wy104 + ((w+x)(y+z)– wy - xz)102 + xz
From four sub-instances to three!
...
...
...
Recursive application
wx
yz
Divide and Conquer
 Result for 2 digits, base 10:
𝑝𝑟𝑜𝑑𝑢𝑐𝑡 = 𝑤𝑦 ⋅ 104 +
𝑤 + 𝑥 𝑦 + 𝑧 – 𝑤𝑦 − 𝑥𝑧 ⋅ 102 + 𝑥𝑧
 Result for 𝑛 digits, base 10:
𝑝𝑟𝑜𝑑𝑢𝑐𝑡 = 𝑤𝑦 ⋅
10𝑛
+
𝑤 + 𝑥 𝑦 + 𝑧 – 𝑤𝑦 − 𝑥𝑧 ⋅
 Result for 𝑛 digits, base 2:
𝑝𝑟𝑜𝑑𝑢𝑐𝑡 = 𝑤𝑦 ⋅
2𝑛
+
𝑤 + 𝑥 𝑦 + 𝑧 – 𝑤𝑦 − 𝑥𝑧 ⋅
𝑛
10 2
𝑛
22
+ 𝑥𝑧
+ 𝑥𝑧
Algorithm
Algorithm
How long does it take?
 Characterize running time using a recurrence
relation.
 Let 𝑡(𝑛) be the amount of time to compute an
answer on an input of size 𝑛.
 Then: 𝑡(𝑛) = 3𝑡
𝑛
2
+ 𝑔(𝑛)
 where 𝑔 𝑛 ∈ 𝑂(𝑛), the cost of dividing and
reassembling sub-results
 Answering a question with a question!
 Want: a closed-form answer
Master Theorem:
Analysis of Divide & Conquer
Define:
𝑎
= number of sub-instances that must be solved
𝑛
= original instance size (variable) *
𝑛/𝑏 = size of sub-instances
𝑑
= polynomial order of 𝑔(𝑛),
where 𝑔(𝑛) = cost of dividing and recombining
t ( n)  a  t ( n / b)  g ( n)
Then:
 O(n d )

t (n)  O(n d log n)
 O(n logb a )

We’ll prove this!
for g (n) O(n )
d
if a  b d  d  log b a
if a  b d  d  log b a
if a  b d  d  log b a
* Assume that 𝑛 is a power of 𝑏.
Analysis
 DC multiplication with Karatsuba’s insight:
 𝑎 = 3: there are 3 instances of multiplication
 𝑏 = 2: each is 1/2 the size of the original multiplication
 𝑑 = 1: the divide and the reassemble has linear complexity
 Thus, 𝑡(𝑛) = 3 ⋅ 𝑡(𝑛 2) + 𝑔(𝑛)
where: 𝑔 𝑛 ∈ 𝑂 𝑛1
 Notice: 𝑎 = 3 > 21 = 𝑏 𝑑
 Recall:
 O(n d )

t (n)  O(n d log n)
 O(n logb a )

 Thus, 𝑡(𝑛) ∈ 𝑂(𝑛log23 ) ≈ 𝑂(𝑛1.585
if a  b d  d  log b a
if a  b d  d  log b a
if a  b d  d  log b a
Another Logarithm Identity
log 𝑚
Consequence:
log 𝑝 𝑛
𝑛 =
log 𝑝 𝑚
Essentials of
Divide and Conquer




Given problem instance of size 𝑛
Divide into 𝑎 sub-instances of size 𝑛/𝑏
Recursively solve these sub-instances
Appropriately combine their answers
 Analyze with the Master Theorem
 In project #2, your job is to apply this pattern
to the Convex Hull problem.
General: Divide and Conquer
function DC (x) : answer
if length(x) < threshold then return adhoc(x)
decompose x into a sub-instances x1, x2 … xa of size n/b
for i  1 to a do yi  DC(xi)
recombine the yi’s to get a solution y for x
return y
Where :
adhoc(x) = is the basic algorithm for small instances
a = the number of divisions at each level
n/b = the fraction of the whole for a sub-instance
Choosing a Threshold
 Threshold = t means:
“stop dividing when problem size = t”
 Threshold depends on
 Algorithm
 and implementation
 and platform
 If t is too big?
 If t is too small?
Recursive vs. Iterative Algorithms
 Tail Recursion can be written iteratively easily
 When results of recursive call aren’t used in an assignment
 In fact, all recursive algorithms can be written as iteration.
How?
 By maintaining your own stack
 Is the iterative implementation better?
 Saves on stack allocation and function call
 Is the recursive implementation better?
 More “elegant” to some tastes:
more natural, more readable, more maintainable.
 Same algorithmic complexity
Next
 Due by end of day Tuesday:
 Screencast #7: Solving Homogeneous
Recurrence Relations directly
 Quiz #2 covering screencast #7
 Due Wednesday: HW #4
 Working up to a proof of the master
theorem