Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Fast Multiplication 1. Polynomial multiplication 2. Toom–Cook method 3. Primitive Roots of Unity 4. The Discrete Fourier Transform 5. The FFT Algorithm 6. Schönhage-Strassen’s method 7. Conclusion 1 1. Polynomial multiplication Polynomial 表示式: p ( x) 5 x 2 x 2 (-5, 1, 2) evaluation 或者 {(-1,-4),(0,-5),(1,-2)} interpolation (p(-1), p(0), p(1)) by Lagrange Interpolating Polynomial ( x 0)( x 1) ( x 1)( x 1) ( x 1)( x 0) 5 2 (1 0)( 1 1) (0 1)(0 1) (1 1)(1 0) 4 2 5 2 2 2 x x x 1 x x 2 1 2 2 x 2 2 x 5 x 2 5 x 2 x p( x) 4 2x2 x 5 2 1. Polynomial multiplication Lagrange Interpolating Polynomial ( x1 , y1 ), ( x2 , y2 ),, ( xm , ym ) p( x) y1 ( x x2 )( x x3 ) ( x xm ) ( x x1 )( x x3 ) ( x xm ) y2 ( x1 x2 )( x1 x3 ) ( x1 xm ) ( x2 x1 )( x2 x3 ) ( x2 xm ) ym ( x x1 )( x x2 ) ( x xm1 ) ( xm x1 )( xm x2 ) ( xm xm 1 ) m m (x x j ) i 1 j 1 ( xi x j ) p ( x ) yi j i 3 1. Polynomial multiplication Polynomial Multiplication Given coefficients (a0,a1,a2,…,an-1) and (b0,b1,b2,…,bn-1) defining two polynomials, p() and q(), and number x, compute p(x)q(x). n 1 C ( x ) p ( x ) q ( x ) ci x i i 0 i ci a j bi j j 0 4 1. Polynomial multiplication Polynomial Multiplication (a0,a1,a2,…,an-1) (b0,b1,b2,…,bn-1) P(x)=a0+a1x+…+an-1xn-1 q(x)=b0+b1x+…+bn-1xn-1 選2n+1個點 (p(w0), p(w1),…, p(w2n) ) (q(w0), q(w1),…, q(w2n) ) evaluation multiplication Polynomial Multiplication (c0,c1,c2,…,cn,…,c2n-1) (p(w0)*q(w0), p(w1)*q(w1),… , p(w2n)*q(w2n) ) interpolation 5 1. Polynomial multiplication Polynomial Multiplication P(x)=5+x q(x)=1+2x (5,1) 選3個點 ( x (p(0)=5, ( x 1)( x 2) 0)( x 2) p(1)=6, ( x 0p(2)=7 )( x 1) ) (1,2)C( x) 5 18 35 (q(0)=1, q(1)=3, (0 1evaluation )(0 2) (1 0)(1 2) (2 0q(2)=5 )( 2 1) ) 5 2 35 2 x 3x 2 18 x 2 2 x x x multiplication 2 2 2 x 2 11x 5 5 11x 2 x 2 Polynomial Multiplication C(0)=5, C(1)=18, C(2)=3 (5,11,2) interpolation 6 1. Polynomial multiplication 慎選特殊的點可以降 低運算 Evaluation O(n) Total O(n2) 7 1. Polynomial multiplication Polynomial Evaluation Horner’s Rule: Given coefficients (a0,a1,a2,…,an-1), defining polynomial n 1 p( x ) ai x i i 0 Given x, we can evaluate p(x) in O(n) time using the equation p( x) a0 x(a1 x(a2 x(an2 xan1 ))) Eval(A,x): [Where A=(a0,a1,a2,…,an-1)] If n=1, then return a0 Else, Let A’=(a1,a2,…,an-1) return a0+x*Eval(A’,x) [assume this can be done in constant time] 8 2.Toom–Cook method Toom-k : a simplification of a description of Toom–Cook polynomial multiplication described by Marco Bodrato in 2007. 給定兩整數M, N,擬計算M*N five main steps: Splitting (將整數轉成多項式) Evaluation (找出多項式對應的點) Pointwise multiplication (點値相乘) Interpolation (由點找出多項式) Recomposition (將多項式轉成整數) Marco Bodrato. Towards Optimal Toom–Cook Multiplication for Univariate and Multivariate Polynomials in Characteristic 2 and 0. In WAIFI'07 proceedings, volume 4547 of LNCS, pages 116-133. June 21-22, 2007. author website 9 2.Toom–Cook method:Toom-3 Toom-3 : 例子說明* M = 12 3456 7890 1234 5678 9012 N= 9 8765 4321 9876 5432 1098 (1) Splitting (將整數轉成多項式) 切成3等份,取base B=108 m0 = 56789012 m1 = 78901234 m2 = 123456 n0 = 54321098 n1 = 43219876 n2 = 98765 p(x) = m0 + m1x + m2x2 q(x) = n0 + n1x + n2x2 10 2.Toom–Cook method:Toom-3 Toom-3 : 例子說明* (2) Evaluation (找出多項式對應的點) p(x) = m0 + m1x + m2x2 q(x) = n0 + n1x + n2x2 p(0) = m0 = 56789012 = 56789012 p(1) = m0 + m1 + m2 = 56789012 + 78901234 + 123456 = 135813702 p(−1) = m0 − m1 + m2 = 56789012 − 78901234 + 123456 = −21988766 p(−2) = m0 − 2m1 + 4m2 = 56789012 − 2×78901234 + 4×123456 = −100519632 p(∞) = m2 = 123456 = 123456 q(0) = n0 = 54321098 = 54321098 q(1) = n0 + n1 + n2 = 54321098 + 43219876 + 98765 = 97639739 q(−1) = n0 − n1 + n2 = 54321098 − 43219876 + 98765 = 11199987 q(−2) = n0 − 2n1 + 4n2 = 54321098 − 2×43219876 + 4×98765 = −31723594 q(∞) = n2 = 98765 = 98765 11 2.Toom–Cook method:Toom-3 Toom-3 : 例子說明* (3) Pointwise multiplication (點値相乘) r(x)=p(x)q(x) r(0) = p(0)q(0) = 56789012 × 54321098 = 3084841486175176 r(1) = p(1)q(1) = 135813702 × 97639739 = 13260814415903778 r(−1) = p(−1)q(−1) = −21988766 × 11199987 = −246273893346042 r(−2) = p(−2)q(−2) = −100519632 × −31723594 = 3188843994597408 r(∞) = p(∞)q(∞) = 123456 × 98765 = 12193131840 12 2.Toom–Cook method:Toom-3 Toom-3 : 例子說明* (4) Interpolation (由點找出多項式) 13 2.Toom–Cook method:Toom-3 Toom-3 : 例子說明* (4) Interpolation (由點找出多項式) 14 2.Toom–Cook method:Toom-3 Toom-3 : 例子說明* (4) Interpolation (由點找出多項式) one sequence given by Bodrato 15 2.Toom–Cook method:Toom-3 Toom-3 : 例子說明* (4) Interpolation (由點找出多項式) 16 2.Toom–Cook method:Toom-3 Toom-3 : 例子說明* (5) Recomposition (將多項式轉成整數) 17 2.Toom–Cook method: Complexity Toom-3 reduces 9 multiplications to 5, and runs in Θ(nlog(5)/log(3)), about Θ(n1.465). In general, Toom-k runs in Θ(c(k)ne), where e = log(2k-1) / log(k), ne is the time spent on sub-multiplications, and c is the time spent on additions and multiplication by small constants n T( n) 5T ( ) O(n) 3 n T( n) 9T ( ) O(n) 3 T( n) O(n logk ( 2 k 1) T( n) O(n2 2 lg n T( n) O(n log3 5 ) ) log n) T( n) O(n1 ) (in Knuth’s book) 18 Polynomial multiplication 慎選特殊的點可以降低運算Primitive Roots of Unity 1, , 2 , 3 , 執行DFT 得到 只需O(nlogn) Evaluation O(n) A(1), A( ), A( 2 ), A( 3 ), Total O(n2) 19 3. Primitive Roots of Unity A number is a primitive n-th root of unity, for n>1, if Example 1: n = 1 The numbers 1, , 2, …, n-1 are all distinct Z*11: x 1 2 3 4 5 6 7 8 9 10 x^2 1 4 9 5 3 3 5 9 4 1 x^3 1 8 5 9 4 7 2 6 3 10 x^4 1 5 4 3 9 9 3 4 5 1 x^5 1 10 1 1 1 10 10 10 1 10 x^6 1 9 3 4 5 5 4 3 9 1 x^7 1 7 9 5 3 8 6 2 4 10 x^8 1 3 5 9 4 4 9 5 3 1 x^9 1 6 4 3 9 2 8 7 5 10 x^10 1 1 1 1 1 1 1 1 1 1 22=4, 62=3, 72=5, 82=9 are 5-th roots of unity in Z*11 2-1=6, 3-1=4, 4-1=3, 5-1=9, 6-1=2, 7-1=8, 8-1=7, 9-1=5 Example 2: The complex number e2pi/n is a primitive n-th root of unity, where i 1 20 3. Primitive Roots of Unity Inverse Property: If is a primitive root of unity, then -1=n-1 Proof: n-1=n=1 n 1 kj 0 j 0 Cancellation Property: For non-zero -n<k<n, n 1 ( k )n 1 ( n )k 1 (1)k 1 11 kj Proof: 0 k k k k 1 1 1 1 j 0 Reduction Property: If w is a primitve (2n)-th root of unity, then 2 is a primitive n-th root of unity. Proof: If 1,,2,…,2n-1 are all distinct, so are 1,2,(2)2,…,(2)n-1 Reflective Property: If n is even, then n/2 = -1. Proof: By the cancellation property, for k=n/2: n 1 0 ( n / 2 ) j 0 n / 2 0 n / 2 0 n / 2 (n / 2)(1 n / 2 ) j 0 Corollary: k+n/2= -k. 21 4.The Discrete Fourier Transform Given coefficients (a0,a1,a2,…,an-1) for an (n-1)-degree polynomial p(x) x0 2 x0 n 1 a 0 p ( x0 ) 1 x0 p( x ) 2 n 1 a 1 x x x 1 1 1 1 1 2 n 1 a p ( x ) n 1 1 x n 1 x n1 x n 1 n 1 The Discrete Fourier Transform is to evaluate p at the values 1,,2,…,n-1 We produce (y0,y1,y2,…,yn-1), where yj=p(j) y 0 p(1) 1 1 y p( ) 1 1 n 1 y n1 p( ) 1 n1 a 0 2 n1 a1 n 1 2 n 1 n 1 a n 1 1 1 22 4.The Discrete Fourier Transform y 0 p(1) 1 1 y p( ) 1 1 n 1 y n1 p( ) 1 n1 a 0 a1 Fa n 1 2 n 1 n 1 a n 1 1 1 2 n 1 Matrix form: y=Fa, where F[i,j]=ij. n 1 y j ai ij i 0 The Inverse Discrete Fourier Transform recovers the coefficients of an (n-1)-degree polynomial given its values at 1,,2,…,n-1 Matrix form: a=F -1y, where F -1[i,j]=-ij/n. 註: the matrix F 中column彼此互相orthogonal滿足 FF*=nI,其中F*為F之conjugate 23 4.The Discrete Fourier Transform The DFT and inverse DFT really are inverse operations Proof: Let A=F -1F. We want to show that A=I, where 1 n 1 ki kj A[i, j ] n k 0 If i=j, then 1 n 1 ki ki 1 n 1 0 1 A[i, i ] n 1 n k 0 n k 0 n If i and j are different, then 1 n 1 ( j i ) k A[i, j ] 0 n k 0 (by Cancellati on Property) 24 4.The Discrete Fourier Transform [a0,a1,a2,...,an-1] [b0,b1,b2,...,bn-1] Pad with n 0's Pad with n 0's [a0,a1,a2,...,an-1,0,0,...,0] [b0,b1,b2,...,bn-1,0,0,...,0] DFT DFT [y0,y1,y2,...,y2n-1] [z0,z1,z2,...,z2n-1] Convolution The DFT and the inverse DFT can be used to multiply two polynomials So we can get the coefficients of the product polynomial quickly if we can compute the DFT (and its inverse) quickly… Component Multiply [y0z0,y1z1,...,y2n-1z2n-1] inverse DFT [c0,c1,c2,...,c2n-1] (Convolution) 25 4.The Discrete Fourier Transform Convolutions 127356 × Convolutions (3,11,37,47,42) 1 2 7 3 5 6 6 12 42 5 10 35 3 6 21 3 11 37 47 42 26 4.The Discrete Fourier Transform Convolutions 127356 × 1 3 2 5 7 6 × 1 3 2 5 7 6 6 12 42 5 10 35 3 6 21 6 12 42 5 10 35 3 6 21 3 11 37 47 42 3 11 37 47 42 + - 3 11 37 50 53 acyclic convolutions 37 (37,50,53) 3 11 44 31 negacyclic convolutions (37,44,31) 27 4.The Discrete Fourier Transform Convolution theorem CyclicConvolution(X, Y) = IDFT(DFT(X) · DFT(Y)) 28 5.The Fast Fourier Transform The FFT is an efficient algorithm for computing the DFT The FFT is based on the divide-and-conquer paradigm: If n is even, we can divide a polynomial into two polynomials and we can write 29 5.The Fast Fourier Transform Given coefficients (a0,a1,a2,…,a7) and w primitive 8-th root of unity 1 1 1 1 M 1 1 1 1 1 1 1 2 4 1 1 4 8 1 6 12 1 1 1 2 1 3 4 1 1 5 6 2 4 6 8 10 12 3 6 9 12 15 18 4 8 12 16 20 24 5 10 15 20 25 30 6 12 18 24 30 36 7 14 21 28 35 42 1 1 a 0 1 6 a 2 3 5 12 a 4 2 6 10 18 44 a6 3 9 15 1 a 0 7 a 1 14 a 2 21 a3 28 a 4 35 a5 42 a 6 a 49 88 7 1 1 1 1 1 1 1 1 1 a1 1 1 1 7 a3 1 2 4 14 a5 1 4 8 21 44 a7 1 6 12 1 1 1 1 1 1 1 2 4 6 3 5 4 8 12 2 6 10 6 12 18 3 9 15 8 16 24 4 12 20 10 20 30 5 15 25 12 24 36 6 18 30 14 28 42 7 21 35 a 0 1 6 a 2 0 12 a 4 0 18 44 a6 0 1 a 0 7 a 2 14 a 4 21 a 6 28 a1 35 a3 42 a5 a 49 88 7 1 0 0 0 1 1 1 0 0 1 2 4 0 2 0 1 4 8 0 0 3 1 6 12 a1 6 a3 12 a5 18 44 a7 1 30 5.The Fast Fourier Transform 1 1 1 2 4 1 1 4 8 1 6 12 1 1 1 2 4 1 1 4 8 1 6 12 1 1 a 0 1 6 a 2 3 5 12 a 4 2 6 10 18 44 a6 3 9 15 1 a0 1 0 0 0 1 6 a 2 0 0 0 1 2 12 a 4 0 0 0 1 3 18 a 44 6 0 0 0 1 1 a1 7 a3 14 a5 21 44 a7 1 1 1 a1 2 4 6 a3 4 8 12 a5 6 12 18 44 a7 1 31 5.The Fast Fourier Transform 1 1 4 1, 1 1 8 16 24 1 1 2 1 1 4 1 6 1 1 4 12 20 10 20 30 5 15 25 12 24 36 6 18 30 14 28 42 7 21 35 4 6 8 12 12 18 a 0 a 2 28 a 4 35 a 6 42 a1 49 88 a3 a 5 a 7 1 1 a 0 1 3 5 a 2 a 2 6 10 4 a6 3 9 15 44 a1 7 a3 14 a5 21 44 a7 1 32 5.The Fast Fourier Transform 令 M n/2 1 1 1 2 4 1 1 4 8 1 6 12 1 6 12 18 a 0 1 a 0 2 M n / 2 a 4 0 a 6 0 M a 0 1 a 0 M 2 n / 2 a 4 0 a 6 0 0 0 0 a1 a 0 0 3 M n / 2 a5 0 2 0 3 0 0 a 7 0 0 0 a1 a 0 0 3 M n/2 a5 0 2 0 3 0 0 a 7 33 5.The Fast Fourier Transform a0 a1 a2 a3 a4 a5 a6 a7 34 5.The Fast Fourier Transform T(n)=2T(n/2)+O(n) The running time is O(n log n). [inverse FFT is similar] 35 Polynomial multiplication Polynomial Multiplication (a0,a1,a2,…,an-1) (b0,b1,b2,…,bn-1) p(x)=a0+a1x+…+an-1xn-1 q(x)=b0+b1x+…+bn-1xn-1 選w primitive m-th root of unity Evaluation FFT Polynomial Multiplication (m2n) (p(w0), p(w1),…, p(wm-1) ) (q(w0), q(w1),…, q(wm-1) ) multiplication (p(w0)*q(w0), p(w1)*q(w1),… , p(wm-1)*q(wm-1) ) (c0,c1,c2,…,cn,…,cm-1) The running time is O(n log n). Interpolation IFFT 36 6.Schönhage-Strassen’s method 給定兩大數M,N。計算M*N (1) Splitting (將整數轉成多項式) 令B=2k,則可產生degree 最多n-1之兩個多項式 p(x)=a0+a1B+…+an-1Bn-1 P=(a0,a1,a2,…,an-1) q(x)=b0+b1B+…+bn-1Bn-1 Q=(b0,b1,b2,…,bn-1) (2) Evaluation (找出多項式對應的點) 選w primitive m-th root of unity (m2n) FFT(P , ) ( p(1), p( ), , p( m 1 )) FFT(Q, ) (q(1), q( ), , q( m 1 )) 37 6. Schönhage-Strassen’s method (3) Pointwise multiplication (點値相乘) (C (1) p(1)q(1), C ( ) p( )q( ), , C ( m1 ) p( m1 )q( m1 )) (4) Interpolation (由點找出多項式) 對上述點執行the inverse Fourier transform C(x) IFFT( (C (1) p(1)q(1), C ( ) p( )q( ), , C ( n1 ) p( m1 )q( m1 )), ) c0 c1 x c 2 n 1 x 2 n2 38 6. Schönhage-Strassen’s method (5)Recomposition (將多項式轉成整數) C(x) c0 c1 x c2n2 x 2n2 令x=B M * N c0 c1B c2n2 B 2 n 2 39 6. Schönhage-Strassen’s method Complexity 分析 The running time is O(n log n) 單位? The bound on the numerical errors α on the ci after the FFT process can be proved to be α ≤ 6n2B2log(n)ε where ε ≅ 1.e-16 the number of digits of these basic numbers should be of the order of log(B)+log(n) 40 6. Schönhage-Strassen’s method Complexity 分析 The running time is O(n log n(log(B) + log(n))2) 然而如果針對所有乘法都採用recursive觀念,即 B1 N , B2 B1 , 得到 T ( N ) N T ( N ) O(n log n) T ( N ) O(n log n log log n) 41 7. Conclusion Method* classical complexity integer O(n 2 ) 1.58 Karatsuba 1962 O(n Toom–3 1963 O(n1.46 ) ) Toom–Cook (knuth) O(n2 2 lg n log n) 1966 SchönhageO(n log n log log n) Strassen 1971 O(log* n) Fürer 2007 O(n log n2 ) 2 215 2 217 10,000位數 40,000位數 42