Download Dynamic Programming

Dynamic Programming
CMPT 438
Second Midterm Exam
• Nov 17 Monday
• Review on Nov 13 Thursday
▫ Master Theorem
▫ Heap, heapsort, priority queue
▫ Dynamic programming
Homework 4
• T(n) = 2T(n/4) + n2
• Find c < 1 so that af(n/b) <= cf(n)
Homework 4
Dynamic Programming
• Not a specific algorithm, but a technique (like
divide-and-conquer).
▫ Used for optimization problems: Find a solution
with the optimal value.
▫ Minimization or maximization.
Two Algorithmic Models:
Divide &
Conquer
Dynamic
Programming
depends on
partitioning
factors
typically small
typically log
function of n
depends on number
and difficulty of
subproblems
View problem as collection of
subproblems
“Recursive” nature
Independent subproblems
Overlapping subproblems
Number of subproblems
Preprocessing
Characteristic running time
Primarily for optimization
problems
Optimal substructure:
optimal solution to problem
contains within it optimal
solutions to subproblems
Four-step method
1. Characterize the structure of an optimal
solution.
2. Recursively define the value of an optimal
solution.
3. Compute the value of an optimal solution,
typically in a bottom-up fashion.
4. Construct an optimal solution from computed
information.
Example: Rod cutting
• You are given a rod of length n ≥ 0 (n in inches)
• A rod of length i inches will be sold for pi dollars
• Cutting is free (simplifying assumption)
• Problem: given a table of prices pi determine the
maximum revenue rn obtainable by cutting up the
rod and selling the pieces.
Length i
Price pi
1
1
2
5
3
8
4
9
5 6 7 8 9 10
10 17 17 20 24 30
If pn is large enough, an optimal solution might require no cuts
Example: Rod Cutting
Length i
Price pi
1
1
2
5
3
8
4
9
5 6 7 8 9 10
10 17 17 20 24 30
Example: Rod Cutting
Step 1: Characterizing an Optimal Solution
Question: in how many different ways can we cut a rod of length n?
For a rod of length 4:
24 - 1 = 23 = 8
For a rod of length n: 2n-1. Exponential: we cannot try all possibilities for n "large".
Example: Rod Cutting
Characterizing an Optimal Solution
Let us find a way to solve the problem recursively:
Let ri be the maximum revenue for a rod of length i.
Assume we have cut a rod of length n into 0 ≤ k ≤ n pieces of length
i1, …, ik,
n = i1 +…+ ik,
with revenue
rn = pi1 + … + pik
Example: Rod Cutting
Characterizing an Optimal Solution
Length i
Price pi
1
1
2
5
3
8
4
9
5 6 7 8 9 10
10 17 17 20 24 30
We begin by constructing (by hand) the optimal solutions for i = 1, …, 10:
r1 = 1 from sln. 1 = 1 (no cuts)
r2 = 5 from sln. 2 = 2 (no cuts)
r3 = 8 from sln. 3 = 3 (no cuts)
r4 = 10 from sln. 4 = 2 + 2
r5 = 13 from sln. 5 = 2 + 3
r6 = 17 from sln. 6 = 6 (no cuts)
r7 = 18 from sln. 7 = 1 + 6 or 7 = 2 + 2 + 3
r8 = 22 from sln. 8 = 2 + 6
r9 = 25 from sln. 9 = 3 + 6
r10 = 30 from sln. 10 = 10 (no cuts)
Example
Example: Rod Cutting
Step 2: Recursively define the value of an optimal solution
Recursion
rn = max(pn, r1 + rn-1, r2 + rn-2, …, rn-1 + r1)
exhibiting optimal substructure
• Optimal solutions to a problem incorporate optimal
solutions to related subproblems.
Reduce to only one subproblem?
rn = max1≤i≤n(pi + rn-i)
Example: Rod Cutting
• Recursive top-down procedure
rn = max1≤i≤n(pi + rn-i)
Problems
• This procedure works, but it is terribly
inefficient.
▫ It could take more than an hour for n = 40.
▫ Running time almost doubles each time n
increases by 1.
• Why so inefficient?
▫ Recomputing overlapping subproblems
Example: Rod Cutting
Dynamic-programming solution
• Solution: solve each subproblem just once
▫ Save the solution to a subproblem in a table
• “Store, don’t recompute” ⇒ time-memory
trade-off.
• exponential-time ⇒ polynomial-time
▫ top-down with memoization
▫ bottom-up
Top-down with memoization
• To find the solution to a subproblem, first look
in the table.
▫ If the answer is there, use it.
▫ Otherwise, compute the solution to the
subproblem and then store the solution in the
table for future use.
• “memoizing” = writing yourself a memo
Recursive vs. Dynamic Programming
rn = max1≤i≤n(pi + rn-i)
Bottom-up
• IDEA: Sort the subproblems by size and solve the
smaller ones first.
rn = max1≤i≤n(pi + rn-i)
Running time
• Both the top-down and bottom-up versions run
in Θ(n2) time.
Subproblem graphs
• Directed graph:
▫ One vertex for each distinct subproblem.
▫ Has a directed edge (x, y)
Reconstructing a solution
• value of an optimal solution
• choices that produced an optimal solution
▫ Extend the bottom-up approach
▫ Save the optimal choices in a separate table
Example: Rod Cutting
Characterizing an Optimal Solution
Length i
Price pi
1
1
2
5
3
8
4
9
5 6 7 8 9 10
10 17 17 20 24 30
We begin by constructing (by hand) the optimal solutions for i = 1, …, 10:
r1 = 1 from sln. 1 = 1 (no cuts)
r2 = 5 from sln. 2 = 2 (no cuts)
r3 = 8 from sln. 3 = 3 (no cuts)
r4 = 10 from sln. 4 = 2 + 2
r5 = 13 from sln. 5 = 2 + 3
r6 = 17 from sln. 6 = 6 (no cuts)
r7 = 18 from sln. 7 = 1 + 6 or 7 = 2 + 2 + 3
r8 = 22 from sln. 8 = 2 + 6
r9 = 25 from sln. 9 = 3 + 6
r10 = 30 from sln. 10 = 10 (no cuts)
Reconstructing a solution
Reconstructing a solution
Reconstructing a solution
Four-step method
1. Characterize the structure of an optimal
solution.
2. Recursively define the value of an optimal
solution.
rn = max1≤i≤n(pi + rn-i)
3. Compute the value of an optimal solution, in a
memoized top down or bottom-up fashion.
4. Construct an optimal solution by extending the
bottom-up process.
31
Example: Matrix-Chain Multiplication
Problem: given a sequence A1, A2, …, An of
matrices, compute the product:
A1  A2  An
• Matrix compatibility:
C=AB
colA = rowB
rowC = rowA
colC = colB
A1  A2  Ai  Ai+1  An
coli = rowi+1
32
Matrix-Chain Multiplication
• In what order should we multiply the matrices?
A1  A2  An
• Matrix multiplication is associative:
• E.g.:
A1  A2  A3 = ((A1  A2)  A3)
= (A1  (A2  A3))
• Which one of these orderings should we choose?
▫ The order in which we multiply the matrices has a
significant impact on the overall cost of executing
the entire chain of multiplications
33
MATRIX-MULTIPLY(A, B)
if columns[A]  rows[B]
then error “incompatible dimensions”
else for i  1 to rows[A]
rows[A]  cols[A]  cols[B]
do for j  1 to columns[B]
multiplications
do C[i, j] = 0
for k  1 to columns[A]
do C[i, j]  C[i, j] + A[i, k] B[k, j]
k
j
i
rows[A]
*
A
j
cols[B]
cols[B]
= i
k
B
C
rows[A]
34
Example
A1  A2  A3
•
•
•
1.
A1: 10 x 100
A2: 100 x 5
A3: 5 x 50
((A1  A2)  A3):
A1  A2 takes 10 x 100 x 5 = 5,000
(its size is 10 x 5)
((A1  A2)  A3) takes 10 x 5 x 50 = 2,500
Total: 7,500 scalar multiplications
2. (A1  (A2  A3)):
A2  A3 takes 100 x 5 x 50 = 25,000
(its size is 100 x 50)
(A1  (A2  A3)) takes 10 x 100 x 50 = 50,000
Total: 75,000 scalar multiplications
one order of magnitude difference!!
35
Matrix-Chain Multiplication
• Given a chain of matrices A1, A2, …, An, where
for i = 1, 2, …, n matrix Ai has dimensions pi-1x pi,
fully parenthesize the product A1  A2  An in a
way that minimizes the number of scalar
multiplications.
A1  A2  Ai  Ai+1  An
p0 x p1 p1 x p2
pi-1 x pi pi x pi+1
pn-1 x pn
36
1. The Structure of an Optimal Parenthesization
• Notation:
Ai…j = Ai Ai+1  Aj, i  j
• For i < j:
Ai…j = Ai Ai+1  Aj
= Ai Ai+1  Ak Ak+1  Aj
= Ai…k Ak+1…j
• Suppose that an optimal parenthesization of Ai…j
splits the product between Ak and Ak+1, where
i
k<j
37
Optimal Substructure
Ai…j = Ai…k Ak+1…j
• The parenthesization of the “prefix” Ai…k must be an
optimal parenthesization
• If there were a less costly way to parenthesize Ai…k, we
could substitute that one in the parenthesization of Ai…j
and produce a parenthesization with a lower cost than
the optimum  contradiction!
• An optimal solution to an instance of the matrix-chain
multiplication contains within it optimal solutions to
subproblems
38
2. A Recursive Solution
• Subproblem:
determine the minimum cost of parenthesizing Ai…j
= Ai Ai+1  Aj
for 1  i  j  n
• Let m[i, j] = the minimum number of
multiplications needed to compute Ai…j
▫ Full problem (A1..n): m[1, n]
▫ i = j: Ai…i = Ai  m[i, i] = 0, for i = 1, 2, …, n
39
2. A Recursive Solution
• Consider the subproblem of parenthesizing
Ai…j = Ai Ai+1  Aj
= Ai…k Ak+1…j
m[i, k]
for 1  i  j  n
pi-1pkpj
for i  k < j
m[k+1,j]
• Assume that the optimal parenthesization splits
the product Ai Ai+1  Aj at k (i  k < j)
m[i, j] = m[i, k]
+
m[k+1, j]
+
pi-1pkpj
min # of multiplications min # of multiplications # of multiplications
to compute Ai…k
to compute Ak+1…j
to compute Ai…kAk…j
40
2. A Recursive Solution
m[i, j] = m[i, k]
+
m[k+1, j]
• We do not know the value of k
+
pi-1pkpj
▫ There are j – i possible values for k: k = i, i+1, …, j-1
• Minimizing the cost of parenthesizing the product
Ai Ai+1  Aj becomes:
0
if i = j
m[i, j] = min {m[i, k] + m[k+1, j] + pi-1pkpj} if i < j
ik<j
41
Example: min {m[i, k] + m[k+1, j] + pi-1pkpj}
m[2, 5] = min
m[2, 2] + m[3, 5] + p1p2p5
m[2, 3] + m[4, 5] + p1p3p5
m[2, 4] + m[5, 5] + p1p4p5
k=2
k=3
k=4
Top-down: Memoized Matrix-Chain
Alg.: MEMOIZED-MATRIX-CHAIN(p)
1.
n  length[p] - 1
2. for i  1 to n
3.
4.
Initialize the m table with
large values that indicate
whether the values of m[i, j]
have been computed
do for j  i to n
do m[i, j]  
5. return LOOKUP-CHAIN(p, 1, n)
Top-down approach
Memoized Matrix-Chain
Alg.: LOOKUP-CHAIN(p, i, j)
1. if m[i, j] < 
2.
then return m[i, j]
3. if i = j
m[i, j] = min {m[i, k] + m[k+1, j] + pi-1pkpj}
ik<j
4.
then m[i, j]  0
5.
else for k  i to j – 1
6.
do q  LOOKUP-CHAIN(p, i, k) +
LOOKUP-CHAIN(p, k+1, j) + pi-1pkpj
7.
if q < m[i, j]
8.
then m[i, j]  q
9. return m[i, j]
44
3. Computing the Optimal Costs
0
if i = j
m[i, j] = min {m[i, k] + m[k+1, j] + pi-1pkpj} if i < j
ik<j
• Length = 1: i = j, i = 1, 2, …, n
• Length = 2: j = i + 1, i = 1, 2, …, n-1
1
m[1, n] gives the optimal
solution to the problem
Compute elements on each
diagonal, starting with the
longest diagonal.
In a similar matrix s we keep the
optimal values of k.
2
3
n
n
j
3
2
1
i
Bottom-up
min ik<j {m[i, k] + m[k+1, j] + pi-1pkpj}
Running time:
O( n3 )
46
Example: min {m[i, k] + m[k+1, j] + pi-1pkpj}
m[2, 2] + m[3, 5] + p1p2p5
m[2, 3] + m[4, 5] + p1p3p5
m[2, 4] + m[5, 5] + p1p4p5
m[2, 5] = min
1
2
3
4
5
k=2
k=3
k=4
6
6
5
4
j
3
2
1
i
• Values m[i, j] depend only
on values that have been
previously computed
47
4. Construct the Optimal Solution
• Store the optimal
choice made at each
subproblem
• s[i, j] = a value of k
such that an optimal
parenthesization of
Ai..j splits the product
between Ak and Ak+1
1
2
3
n
n
k
j
3
2
1
i
48
4. Construct the Optimal Solution
• s[1, n] is associated with
the entire product A1..n
▫ The final matrix
multiplication will be split
at k = s[1, n]
A1..n = A1..k  Ak+1..n
A1..n = A1..s[1, n]  As[1, n]+1..n
▫ For each subproduct
recursively find the
corresponding value of k
that results in an optimal
parenthesization
1
2
3
n
n
j
3
2
1
i
49
4. Construct the Optimal Solution
• s[i, j] = value of k such that the optimal
parenthesization of Ai Ai+1  Aj splits the
product between Ak and Ak+1
1
2
3
4
5
6
6
3
3
3
5
5
-
5
3
3
2
-
3
3
-
4
-
-
2
3
3
1
1
1
-
4
3
• s[1, n] = 3  A1..6 = A1..3 A4..6
• s[1, 3] = 1  A1..3 = A1..1 A2..3
• s[4, 6] = 5  A4..6 = A4..5 A6..6
j
i
50
4. Construct the Optimal Solution
PRINT-OPT-PARENS(s, i, j)
1
2
3
4
5
6
6
3
3
3
5
5
-
then print “A”i
5
4
3
3
-
4
-
2
3
3
2
-
-
else print “(”
3
3
1
1
1
-
if i = j
PRINT-OPT-PARENS(s, i, s[i, j])
PRINT-OPT-PARENS(s, s[i, j] + 1, j)
print “)”
3
i
j