* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Dynamic Programming
Survey
Document related concepts
Lateral computing wikipedia , lookup
Computational electromagnetics wikipedia , lookup
Algorithm characterizations wikipedia , lookup
Genetic algorithm wikipedia , lookup
Computational complexity theory wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Multiple-criteria decision analysis wikipedia , lookup
Dijkstra's algorithm wikipedia , lookup
Corecursion wikipedia , lookup
Knapsack problem wikipedia , lookup
Selection algorithm wikipedia , lookup
Multi-objective optimization wikipedia , lookup
Expectation–maximization algorithm wikipedia , lookup
Factorization of polynomials over finite fields wikipedia , lookup
Transcript
Designing a Dynamic Programming Algorithm for an Optimization Problem • Step 1. Characterize the structure of optimal solution • Step 2. Recursively define the value of an optimal solution • Step 3. Compute the value of an optimal solution in a bottombottom-up fashion • Step 4. Construct an optimal solution from computed information Dynamic Programming: Matrix Chain Multiplication Matrix Chain Multiplication Problem Given a chain A1 , A2 , … , An of n matrices, where (for i = 1, 2,… , n) A is a pi × pi i matrix Fully parenthesize the product A1 A2 Time Complexity of Matrix Multiplication pxq qxr Matrix-Multiply(A, B) if columns[A] ≠ rows[B] then error “incompatible dimensions” pqr else for iÅ1 to rows[A] = p do {{ rq for j Å 1 to columns[B] = r do C[I, j] Å 0 q for k Å 1 to columns[A] = q do C[I, j] Å C[I, j] + A[I, k]*B[k, j] An so that the number of scalar multiplications is minimum Example Two ways to parenthesize ( A1 10 100 A2 ) 5 A1 10 A3 50 A2 100 A1 ( A2 10 100 A3 5 50 A3 ) 5 50 = (10)(100)(5) + (10)(5)(50) = (100)(5)(50) + (10)(100)(50) = 5000 + 2500 = 7500 = 25000 + 50000 = 75000 { return C 1 A1 A2 A3 A4 Let P ( n ) = number of ways to parenthesize a product of n matrices, then Can be parenthesized in 5 ways ( A ( A ( A A ))) 1 2 3 if n = 1 1 n P ( n) = ∑ P ( k ) P ( n − k ) if n ≥ 2 k =1 It can be shown that P ( n ) = C ( n − 1) , where C ( n − 1) is the (n-1)-th Catalan 4 ( A (( A A ) A )) 1 2 3 4 ( ( A1 A2 ) ( A3 A4 ) ) (( A ( A A )) A ) 1 2 3 number 4 ( ( ( A1 A2 ) A3 ) A4 ) Notation m[i, j] = Ai… j = Ai Ai + 1 C ( n) = Objective Aj Min # of mult. to evaluate 1 2n 4n = Ω 3/2 n + 1 n n Ai… j if 0 m[ i , j ] = min m + mk +1, j + pi −1 pk p j ) if i ≤ k ≤ j ( i ,k i= j i< j Find a way to fully parenthesize the product A1…n = A1 A2 An in such a way that the cost is minimum s[i, j] = that k such that m [ i , j ] = m [ i , k ] + m [ k + 1, j ] + pi −1 pk p j 2 • Optimal Substruc. Property ( --- ) ( --- ) •Overlapping Subproblems Property Recursive formula for m[I, j] leads to an exponential problem Observation: Relatively few problems Set of problems { i…j } So # of subprobs. is the # of ways of choosing I, j such that 1 ≤ i ≤ j ≤ n , which is n 2 + n ⇒ Overlap (( ) )( ( ) ) 3 Analysis of RecursiveRecursive-MatrixMatrix-Chain T ( n) = time to compute optimal parenth. of chain of n matrices T (1) ≥ 1 n −1 T ( n ) 1 ≥ + ( T ( k ) + T ( n − k ) + 1) ∑ k =1 T (n) ≥ 1 + n −1 ∑ (T ( k ) + T ( n − k ) + 1 ) k =1 n −1 = 2∑ T (k ) + n k =1 We use math induction to prove that T ( n) = Ω ( 2 n ) The basis step is easy: T (1) ≥ 1 = 2 0 Assuming the Induction hypothesis for ≤ n − 1, we have: n −1 n− 2 k =1 k =0 for n>1 T( ) ≥ 2 −1 T ( n ) ≥ 2∑ 2 k − 1 + n = 2∑ 2 k + n = 2( 2 n −1 − 1) + n = 2n − 2 + n ≥ 2n −1 4 Memoization Memoization • Create a table for solns to subprobs so far computed • Initialize each table entry to a symbol indicating subprob not yet computed • Each time a subproblem is first encountered, its soln is computed and placed in the table • Each subsequent time it is encountered, its previously computed soln is simply retrieved from the table How to Design a Dynamic Prog. Prog. Algorithm 1. Identify the subproblems – • Subprobs must be of same type as original probs How to Design a Dynamic Prog. Prog. Algorithm 3. Apply “generic dynamic prog. algorithm – • Subprobs must be smaller • Use bottom-up version • How to specify each subprob? • use top-down version (Memoization) •What is the base case? What is the recursive case? 2. Characterize the op. soln as a combination of op solns to subprobs – • Focus on value of soln rather than description of soln 4. Modify your algorithm to compute description of soln to value of soln, if needed • Add additional table • Introduce notation to rep. value of op soln to each subprob • Consider trying all possible solns to prob 5 Longest Common Subsequence Longest Common Subsequence (LCS) If { Def. A seq X = A, B , C , B , D, A, B increasing seq such that xi = then common subsequences are, for example, and is a subseq of X = x1 , x2, … , xn if there is a strictly Y = B , D, C , A, B , A BCA Z = z1 , z2, … , zk (LCS) j i1 , i2 ,… , ik zj of the indices of X LCS Problem: Given two seqs, find an LCS BCBA Both are common subsequences (CS). But only the last is a LCS. Notation: Given X = x1 , x2, … , xn , the i-th prefix X i is defined as the subseq x1 , x2, … , xi Thm 16 (p351). (Optimal substructure of LCS) Let X = x1 , x2 ,… , xm be seqs, Y = y , y , … , y 1 2 n and let 1. Z = z1, z2,…, zk = LCS ( X,Y ) xm = yn ⇒zk = xm = yn xm ≠ yn 2. & ⇒ z k ≠ x m 3. xm ≠ y & zk ≠ yn n ⇒ and So … , then ? xm = yn Zk−1 = LCS( Xm−1,Yn−1) Z = LC S (X m − 1 Z = LC S (X ,Y ,Y n − 1 ) ) Yes . . zk = xm = yn & Zk−1 = LCS( Xm−1,Yn−1 ) NO ? zk ≠ x m & Z = LCS ( X m −1 , Y ) z k ≠ yn & Z = LCS ( X ,Yn−1 ) 6 Notation c[i, j] = Length of LCS ( X i ,Y j ) 0 if i =0or j = 0 c[ i, j] = c[ i −1, j −1] +1 if i, j >0&xi = yj Max( c[ i, j −1] ,c[ i −1, j]) if i, j >0&xi ≠ yj [ [ ] b i , j points to the entry of c 0… i ,0… corresponding to the optimal subprob soln chosen when computing c i , j [ j] ] 7