Survey

* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project

Survey

Document related concepts

Transcript

Recurrence Relations and Recursive Algorithms For a numeric function (a0, a1, a2, ..., ar, ...), an equation relating ar, for any r, to one or more of the ai’s, i < r, is called a recurrence relation. A recurrence relation is also called a difference equation. Boundary conditions: The value of function at one or more points is given. A recurrence relation together with an appropriate set of boundary conditions can describe a numeric function. The numeric function is also referred to as the solution of the recurrence relation. Unfortunately, no general method of solution for handling all recurrence relations is known. kth-order linear recurrence relation with constant coefficients : A recurrence relation of the form C0ar+ C1ar-1+ C2ar-2+...+ Ck ar-k = f(r), (*) where Ci’s are constants, C0 0 and Ck 0. Example : 1st-order: 2ar + 3ar-1 = 2r 2nd-order: 3ar - 5ar-1 + 2ar-2= r2 + 5 1 Case I: If k consecutive values of the numeric function a, am-k, am-k+1,..., am-1 are known for some m, the value of am can be calculated am=-1/C0[C1am-1+ C2am-2+...+ Ckam-k - f(m)]. am+1: am+1=-1/C0[C1am+ C2am-1+...+ Ckam-k+1 - f(m+1)] the values of am+2, am+3,...can be computed similarly. am-k-1: am-k-1=-1/Ck[C0am-1+ C1am-2+...+ Ck-1am-k - f(m-1)] am-k-2: am-k-2=-1/Ck[C0am-2+ C1am-3+...+ Ck-1am-k-1 - f(m-2)] The value of am-k-3, am-k-4,...can be computed similarly. Case II: Fewer than k values of the numeric function will not be sufficient to determine the numeric function uniquely. Case III: More than k values of the numeric function might make it possible for the existence of a numeric function that satisfies the recurrence relation and the given boundary condition. 2 Homogeneous Solutions The solution of a linear difference equation with constant coefficients is the sum of two parts, the homogeneous solution, which satisfies the difference equation when the right-hand side of the equation is set to 0, and the particular solution, which satisfies the difference equation with f(r) on the right-hand side. Total solution = homogeneous solution + particular solution. homogeneous solution: a(h)=(a0(h), a1(h),..., ar(h),...) particular solution: a(p)=(a0(p), a1(p),..., ar(p),...). Since C0ar(h)+ C1ar-1(h)+...+ Ckar-k(h) = 0 and C0ar(p)+ C1ar-1(p)+...+ Ckar-k(p) = f(r) We have C0(ar(h)+ar(p))+ C1(ar-1(h)+ar-1(p))+...+ Ck(ar-k(h)+ar-k(p))= f(r) The total solution, a = a(h)+a(p) satisfies the difference equation. characteristic root solution: A homogeneous solution of a linear difference equation with constant coefficients is of the r form Aα1 , whereα1 is called a characteristic root and A is a constant determined by the boundary conditions. r characteristic equation: Substituting Aα1 for ar, we have r r-1 r-2 r-k C0 Aα + C1 Aα + C2 Aα +...+ Ck Aα = 0 3 It can be simplified to r r-1 r-2 r-k C0α + C1α + C2α +...+ Ckα = 0 r Ifα1 is one of the roots of the equation, Aα1 is a homogeneous solution to the difference equation. multiple roots: Letα1 be a root of multiplicity m. The corresponding homogeneous solution becomes (A1rm-1+ A2rm-2+ ...+ Am-2r2+ Am-1r1+ Am)α1r, where the Ai’s are constants to be determined by the boundary conditions. Am-1rα1r is also a homogeneous solution: Recall thatα1 not only is a root of the equation C0αr+ C1αr-1+ C2αr-2+...+ Ckαr-k = 0 (**) But also is a root of the derivative equation of (**), C0 rαr-1+ C1 (r-1)αr-2+ C2 (r-2)αr-3+...+ Ck (r-k)αr-k-1 = 0 (***) Becauseα1 is a multiple root of (**). Multiplying (***) by Am-1α and replacingαbyα1, we obtain C0 Am-1rα1+C1 Am-1(r-1)α1r-1+C2 Am-1(r-2)α1r-2+ ...+ Ck Am-1(r-k)α1r-k = 0 4 Particular Solutions There is no general procedure for determining the particular solution of a difference equation. Example : Consider the difference equation ar - 5ar-1 + 6ar-2= 3r2 (*) we assume that the general form of the particular solution is P1r2 + P2r + P3 Substituting it into the left-hand side of (*), we obtain P1r2 + P2r + P3 + 5P1(r – 1)2 + 5P2(r – 1) + 5P3 + 6P1(r – 2)2 + 6P2(r – 2) + 6P3 Which simplifies to 12P1r2 – (34P1 – 12P2)r + (29P1 - 17P2 + 12P3) (**) Comparing (**) with the right-hand side of (*), we obtain 12P1 = 3 34P1 – 12P2 = 0 29P1 - 17P2 + 12P3 = 0, which yield P1 = 1/4, P2 = 17/24, P3 = 115/288 In general, when f(r) is of the form of a polynomial of degree t in r F1rt + F2rt-1 + ... + Ftr +Ft+1. The corresponding particular solution will be of the form P1rt + P2rt-1 + ... + Ptr +Pt+1. 5 Total Solutions For a kth-order difference equation, the k undetermined coefficients A1, A2,..., Ak in the homogenous solution can be determined by the boundary conditions, aro, aro+1,...aro+k-1,for any r0. Suppose the characteristic roots of the difference equation are all distinct. The total solution is of the form r r r ar= A1α1 + A2α2 +...+ Akαk +p(r) where p(r) is the particular solution. Thus, for r = r0, r0+1,..., r0+k-1,we have the system of linear equations: r0 r0 r0 ar0= A1α1 + A2α2 +...+ Akαk +p(r0) r0+1 r0+1 r0+1 ar0+1= A1α1 + A2α2 +...+ Akαk +p(r0+1) ........................ + A2α2r0+k-1+...+ Akαkr0+k-1+p(r0+k-1) r0+k-1 1 ar0+k-1= A1α These k linear equation can be solved for A1, A2,..., Ak. 6 Solution by the Method of Generating Functions Consider the recurrence realtion ar = 3ar-1 + 2 r 1 With the boundary condition a0 = 1. Multiplying both sides by zr and summing for all r, r 1, r 1 r 1 r 1 ar z r 3 ar 1 z r 2 z r a z r 1 r r A( z ) a 0 r 1 r 1 ar 1 z r z ar 1 z r 1 zA( z ) We obtain A( z ) a0 3zA( z ) A( z ) 2z 1 z 1 z 2 1 (1 3z )(1 z ) 1 3z 1 z Consequently, we have ar = 23r – 1, r 0. 7 Sorting Algorithms Recursive specification of BUBBLESORT: Let S(x1,x2,...,xn) denote BUBBLESORT algorithm that sorts the numbers in registers x1, x2,..., xn in ascending order, let M(x1, x2,..., xn) denote LARGEST2 algorithm that places in register xn the largest number among the numbers in registers x1, x2, ..., xn. We have S(x1, x2,..., xn) M(x1, x2,..., xn) S(x1, x2,..., xn-1) The symbol “” means “is defined to be”. M(x1 x2,..., xn) can be defined as M(x1, x2,..., xn) A(x1, x2)A(x2, x3) ... A(xn-2, xn-1)A(xn-1, xn). S(x1, x2,..., xn) can be rewritten as S(x1, x2,..., xn) A(x1, x2)A(x2, x3) ... A(xn-2, xn-1)A(xn-1, xn) S(x1, x2,..., xn-1). With the boundary condition S(x1, x2) A(x1, x2) Let an denote the number of comparison steps the bobble sort algorithm takes to sort n numbers. We have, an = (n-1) + an-1, using the boundary condition a1 = 0, we obtain an = n(n-1)/2. 8 Bose-Nelson algorithm, (T(x1, x2,..., xn), n = 2r): Let P[(x1, x2,..., xn), (xn+1, xn+2,..., x2n)] denote an algorithm that will arrange the number in registers x1, x2,..., x2n in ascending order giving that the numbers x1, x2,..., xn have already been arranged in ascending order, as have the numbers in registers xn+1, xn+2,..., x2n. T(x1, x2,..., x2n) T(x1, x2,..., xn) T(xn+1, xn+2,..., x2n) P[(x1, x2,..., xn), (xn+1, xn+2,..., x2n)] (*) The algorithm P[(x1, x2,..., xn), (xn+1, xn+2,..., x2n)] can be defined recursively as: P[(x1,..., xn), (xn+1,..., x2n)] P[(x1,..., xn/2), (xn+1,..., x3n/2)] P[(xn/2+1,..., xn), (x3n/2+1,..., x2n)] P[(xn/2+1,..., xn), (xn+1,..., x3n/2)] (**) 1 (n/2) (n/2)+1 n n+1 (3n/2) (3n/2)+1 2n We can express T(x1, x2,..., xn),n = 2r,as a sequence of procedures of the forms T(xI, xj) and P[(xi), (xj)], both of T(xi, xj) and P[(xi), (xj)] are equal to A(xi, xj). Let cr denote the number of comparison steps that P[(x1, x2,..., x2rr), (x2rr+1, x2rr+2,..., x2rr+r)] takes. According to the relation in (**), we have cr = 3cr-1. Solving it with the boundary condition c0 = 1,we obtain cr = 3r. 9 Let dr denote the number of comparison steps the procedure T(x1, x2,..., xn) takes. According to the relation in (*),we have dr = 2dr-1+cr-1 or dr =2dr-1+3r-1 we obtain dr = B2r+3r From the boundary condition d0 = 0, we have B = -1. Thus, Dr = 3r - 2r. odd-even merge algorithm, U(x1, x2,..., xn): Let U(x1, x2,..., x2n) U(x1, x2,..., xn) U(xn+1, xn+2,..., x2n) B[(x1, x2, ..., xn), (xn+1, xn+2,..., x2n)], where the algorithm B is defined to be B[(x1, x2, x3, x4, ..., x2n), (y1, y2, y3, y4, ..., y2n)] B[(x1, x3, x5, ..., x2n-1), (y1, y3, y5, ..., x2n-1)] B[(x2, x4, x6, ..., x2n), (y2, y4, y6, ..., x2n)] A(x2, x3) A(x4, x5) A(x6, x7) ... A(x2n, y1) A(y2,y3) A(y4,y5)... A(y2n-2,y2n-1) (*) To verify the validity of (*), we want first to show that after the execution of B[(x1, x3, x5, ..., x2n-1), (y1, y3, y5, ..., x2n-1)] and B[(x2, x4, x6, ..., x2n), (y2, y4, y6, ..., x2n)] 1. The number in x1 is the smallest of the 4n numbers. 2. The number in y2n is the largest of the 4n numbers. 3. The two numbers in x2i and x2i+1,1≦ i ≦ n-1,are the 2ith and the (2i+1)st smallest of the 4n numbers, the two numbers in x2n and y1 are the 2nth and (2n+1)st smallest of the 4n numbers, and the two numbers in y2i and y2i+1, 1≦ i ≦ n - 1, are the (2n +2i)th and (2n+2i+1)st smallest of the 4n numbers. 10 If this is indeed the case, the comparison A(y2, y3) A(y4, y5) ... A(y2n-2, y2n-1) will yield a sorted sequence of 4n numbers. Let cr denote the number of comparison steps the algorithm B[(x1, x2, x3, x4, ..., x2n), (y1, y2, y3, y4, ..., yn)] takes. According to the relation in (*), we have cr = 2cr-1 + (2r-1) with the boundary condition c0 = 1, we obtain cr = r2r +1. Let dr denote the number of comparison steps the algorithm U(x1, x2, ..., xn) takes. According to the relation in (*), we have dr = 2dr-1 + cr-1 or dr = 2dr-1+(r-1)2r-1 +1 with boundary condition d0 = 0, we obtain dr = (r2-r+4)2r-2 – 1. 11 Matrix Multiplication Algorithms Given two n n matrices A and B, we can partition them into (n/2) (n/2) submatrices: a A 11 a21 a12 b11 B b a22 21 b12 b22 The product AB can be expressed as c c C AB 11 12 c21 c22 Where a11, a12, a21, a22, b11, b12, b21, b22, c11, c12, c21, c22 are all (n/2) (n/2) submatrices. We can then compute m1=(a11+ a22) (b11+ b22) m2=(a12 - a22) (b21 + b22) m3=(a11 - a21) (b11 + b12) m4=(a11 + a12) b22 m5=(a21 + a22) b11 m6=a11(b12 - b22) m7=a22(b21 - b11) Finally, we can compute c11=m1 + m2 - m4 + m7 c12=m4 + m6 c21=m5 + m7 c22=m1 - m3 - m5 + m6. 12 The total cost of multiplying A and B is 7 matrix multiplications and 18 matrix additions. If we use the classical method to carry out multiplication of (n/2)(n/2) matrices, the total number of arithmetic operations will be 2 n 3 n 2 7 11 n 7 2 18 n3 n 2 4 4 2 2 2 Let fr denote the total number of arithmetic operations required to multiply two 2r2r matrices. We have fr = 7 fr-1+ 18 ·22r-2 r ≧ 1 with the boundary condition f0 = 1, we obtain fr = 7 ·7r - (18/3) ·4r Thus, for n = 2r fr = 7 ·7lgn - (18/3) ·n2 = 7n2.81 - (18/3) ·n2. Since the time complexity of the algorithm presented above is O (n2.81), the time complexity of the matrix multiplication problem is O(n2.81). 13