Download Document

Fast Multiplication
1. Polynomial multiplication
2. Toom–Cook method
3. Primitive Roots of Unity
4. The Discrete Fourier Transform
5. The FFT Algorithm
6. Schönhage-Strassen’s method
7. Conclusion
1
1. Polynomial multiplication

Polynomial 表示式:
p ( x)  5  x  2 x 2
(-5, 1, 2)
evaluation
或者
{(-1,-4),(0,-5),(1,-2)}
interpolation
(p(-1), p(0), p(1))
by Lagrange Interpolating Polynomial
( x  0)( x  1)
( x  1)( x  1)
( x  1)( x  0)
 5
 2
(1  0)( 1  1)
(0  1)(0  1)
(1  1)(1  0)
4 2
5 2
2 2

x x 
x 1 
x x
2
1
2
 2 x 2  2 x  5 x 2  5  x 2  x
p( x)  4


 2x2  x  5




2
1. Polynomial multiplication
Lagrange Interpolating Polynomial
( x1 , y1 ), ( x2 , y2 ),, ( xm , ym )
p( x)  y1
( x  x2 )( x  x3 )  ( x  xm )
( x  x1 )( x  x3 )  ( x  xm )
 y2

( x1  x2 )( x1  x3 )  ( x1  xm )
( x2  x1 )( x2  x3 )  ( x2  xm )
  ym
( x  x1 )( x  x2 )  ( x  xm1 )
( xm  x1 )( xm  x2 )  ( xm  xm 1 )
m
m
(x  x j )
i 1
j 1
( xi  x j )
p ( x )   yi 
j i
3
1. Polynomial multiplication
Polynomial Multiplication

Given coefficients (a0,a1,a2,…,an-1) and
(b0,b1,b2,…,bn-1) defining two polynomials, p() and
q(), and number x, compute p(x)q(x).
n 1
C ( x )  p ( x ) q ( x )   ci x
i
i 0
i
ci   a j bi  j
j 0
4
1. Polynomial multiplication
Polynomial Multiplication
(a0,a1,a2,…,an-1)
(b0,b1,b2,…,bn-1)
P(x)=a0+a1x+…+an-1xn-1
q(x)=b0+b1x+…+bn-1xn-1
選2n+1個點 (p(w0), p(w1),…, p(w2n) )
(q(w0), q(w1),…, q(w2n) )
evaluation
multiplication
Polynomial Multiplication
(c0,c1,c2,…,cn,…,c2n-1)
(p(w0)*q(w0), p(w1)*q(w1),…
, p(w2n)*q(w2n) )
interpolation
5
1. Polynomial multiplication
Polynomial Multiplication
P(x)=5+x
q(x)=1+2x
(5,1)
選3個點 ( x (p(0)=5,
( x  1)( x  2)
0)( x  2) p(1)=6,
( x  0p(2)=7
)( x  1) )
(1,2)C( x)  5
 18

35
(q(0)=1,
q(1)=3,
(0  1evaluation
)(0  2)
(1  0)(1  2)
(2  0q(2)=5
)( 2  1) )

 



5 2
35 2
x  3x  2  18 x 2  2 x 
x  x multiplication
2
2
 2 x 2  11x  5  5  11x  2 x 2

Polynomial Multiplication
C(0)=5, C(1)=18, C(2)=3
(5,11,2)
interpolation
6
1. Polynomial multiplication
慎選特殊的點可以降
低運算
Evaluation O(n)
Total O(n2)
7
1. Polynomial multiplication
Polynomial Evaluation

Horner’s Rule:

Given coefficients (a0,a1,a2,…,an-1), defining polynomial
n 1
p( x )   ai x i
i 0

Given x, we can evaluate p(x) in O(n) time using the equation
p( x)  a0  x(a1  x(a2    x(an2  xan1 )))

Eval(A,x):


[Where A=(a0,a1,a2,…,an-1)]
If n=1, then return a0
Else,


Let A’=(a1,a2,…,an-1)
return a0+x*Eval(A’,x)
[assume this can be done in constant time]
8
2.Toom–Cook method
Toom-k :
a simplification of a description of Toom–Cook polynomial
multiplication described by Marco Bodrato in 2007.
給定兩整數M, N，擬計算M*N
five main steps:
Splitting (將整數轉成多項式)
Evaluation (找出多項式對應的點)
Pointwise multiplication (點値相乘)
Interpolation (由點找出多項式)
Recomposition (將多項式轉成整數)
Marco Bodrato. Towards Optimal Toom–Cook Multiplication for Univariate and Multivariate Polynomials in Characteristic 2
and 0. In WAIFI'07 proceedings, volume 4547 of LNCS, pages 116-133. June 21-22, 2007. author website
9
2.Toom–Cook method:Toom-3
Toom-3 : 例子說明*
M = 12 3456 7890 1234 5678 9012
N= 9 8765 4321 9876 5432 1098
(1) Splitting (將整數轉成多項式)
切成3等份，取base B=108
m0 = 56789012
m1 = 78901234
m2 = 123456
n0 = 54321098
n1 = 43219876
n2 = 98765
p(x) = m0 + m1x + m2x2
q(x) = n0 + n1x + n2x2
10
2.Toom–Cook method:Toom-3
Toom-3 : 例子說明*
(2) Evaluation (找出多項式對應的點)
p(x) = m0 + m1x + m2x2
q(x) = n0 + n1x + n2x2
p(0) = m0 = 56789012 = 56789012
p(1) = m0 + m1 + m2 = 56789012 + 78901234 + 123456 = 135813702
p(−1) = m0 − m1 + m2 = 56789012 − 78901234 + 123456 = −21988766
p(−2) = m0 − 2m1 + 4m2 = 56789012 − 2×78901234 + 4×123456 = −100519632
p(∞) = m2 = 123456 = 123456
q(0) = n0 = 54321098 = 54321098
q(1) = n0 + n1 + n2 = 54321098 + 43219876 + 98765 = 97639739
q(−1) = n0 − n1 + n2 = 54321098 − 43219876 + 98765 = 11199987
q(−2) = n0 − 2n1 + 4n2 = 54321098 − 2×43219876 + 4×98765 = −31723594
q(∞) = n2 = 98765 = 98765
11
2.Toom–Cook method:Toom-3
Toom-3 : 例子說明*
(3) Pointwise multiplication (點値相乘)
r(x)=p(x)q(x)
r(0) = p(0)q(0) = 56789012 × 54321098 = 3084841486175176
r(1) = p(1)q(1) = 135813702 × 97639739 = 13260814415903778
r(−1) = p(−1)q(−1) = −21988766 × 11199987 = −246273893346042
r(−2) = p(−2)q(−2) = −100519632 × −31723594 = 3188843994597408
r(∞) = p(∞)q(∞) = 123456 × 98765 = 12193131840
12
2.Toom–Cook method:Toom-3
Toom-3 : 例子說明*
(4) Interpolation (由點找出多項式)
13
2.Toom–Cook method:Toom-3
Toom-3 : 例子說明*
(4) Interpolation (由點找出多項式)
14
2.Toom–Cook method:Toom-3
Toom-3 : 例子說明*
(4) Interpolation (由點找出多項式)
one sequence given by Bodrato
15
2.Toom–Cook method:Toom-3
Toom-3 : 例子說明*
(4) Interpolation (由點找出多項式)
16
2.Toom–Cook method:Toom-3
Toom-3 : 例子說明*
(5) Recomposition (將多項式轉成整數)
17
2.Toom–Cook method:
Complexity
Toom-3 reduces 9 multiplications to 5, and runs in Θ(nlog(5)/log(3)), about
Θ(n1.465). In general, Toom-k runs in Θ(c(k)ne), where e = log(2k-1) / log(k),
ne is the time spent on sub-multiplications, and c is the time spent on
additions and multiplication by small constants
n
T( n)  5T ( )  O(n)
3
n
T( n)  9T ( )  O(n)
3
T( n)  O(n
logk ( 2 k 1)
T( n)  O(n2
2 lg n
T( n)  O(n log3 5 )
)
log n)
T( n)  O(n1 )
(in Knuth’s book)
18
Polynomial multiplication
慎選特殊的點可以降低運算Primitive Roots of Unity
1,  ,  2 ,  3 , 
執行DFT 得到
只需O(nlogn)
Evaluation O(n)
A(1), A( ), A( 2 ), A( 3 ),
Total O(n2)
19
3. Primitive Roots of Unity

A number  is a primitive n-th root of unity, for n>1, if



Example 1:




n = 1
The numbers 1, , 2, …, n-1 are all distinct
Z*11:
x
1
2
3
4
5
6
7
8
9
10
x^2
1
4
9
5
3
3
5
9
4
1
x^3
1
8
5
9
4
7
2
6
3
10
x^4
1
5
4
3
9
9
3
4
5
1
x^5
1
10
1
1
1
10
10
10
1
10
x^6
1
9
3
4
5
5
4
3
9
1
x^7
1
7
9
5
3
8
6
2
4
10
x^8
1
3
5
9
4
4
9
5
3
1
x^9
1
6
4
3
9
2
8
7
5
10
x^10
1
1
1
1
1
1
1
1
1
1
22=4, 62=3, 72=5, 82=9 are 5-th roots of unity in Z*11
2-1=6, 3-1=4, 4-1=3, 5-1=9, 6-1=2, 7-1=8, 8-1=7, 9-1=5
Example 2: The complex number e2pi/n is a primitive n-th root of
unity, where i   1
20
3. Primitive Roots of Unity

Inverse Property: If  is a primitive root of unity, then  -1=n-1


Proof: n-1=n=1
n 1
kj

 0
j 0
Cancellation Property: For non-zero -n<k<n,
n 1
( k )n  1 ( n )k  1 (1)k  1
11
kj

Proof:





0

k
k
k
k
 1
 1
 1  1
j 0
Reduction Property: If w is a primitve (2n)-th root of unity, then 2 is
a primitive n-th root of unity.


Proof: If 1,,2,…,2n-1 are all distinct, so are 1,2,(2)2,…,(2)n-1
Reflective Property: If n is even, then n/2 = -1.

Proof: By the cancellation property, for k=n/2:
n 1
0   ( n / 2 ) j   0   n / 2   0   n / 2     0   n / 2  (n / 2)(1   n / 2 )
j 0

Corollary: k+n/2= -k.
21
4.The Discrete Fourier Transform


Given coefficients (a0,a1,a2,…,an-1) for an (n-1)-degree polynomial
p(x)
x0 2  x0 n 1  a 0 
 p ( x0 )  1 x0

 p( x )  

2
n 1
a


1
x
x

x
1
1
1
 1 
 1 



 
 






2
n 1  a

p
(
x
)
n 1 

1 x n 1 x n1  x n 1   n 1 
The Discrete Fourier Transform is to evaluate p at the values

1,,2,…,n-1

We produce (y0,y1,y2,…,yn-1), where yj=p(j)
 y 0  p(1)
 1 1
 y  p( )
 1 
 1


 

 
n 1
 y n1  p( ) 1  n1
 a 0 

2 
 n1  a1 


  


n 1 2
n 1 n 1  a
n

1


 

1


1


22
4.The Discrete Fourier Transform
 y 0  p(1)
 1 1
 y  p( )
 1 
1



 

 
n 1
 y n1  p( ) 1  n1

 a 0 


 

 a1 
    Fa


n 1 2
n 1 n 1  a
n

1



 


1
1
2
n 1



Matrix form: y=Fa, where F[i,j]=ij.
n 1
y j   ai ij
i 0

The Inverse Discrete Fourier Transform recovers the coefficients of
an (n-1)-degree polynomial given its values at 1,,2,…,n-1

Matrix form: a=F -1y, where F -1[i,j]=-ij/n.
註: the matrix F 中column彼此互相orthogonal滿足
FF*=nI，其中F*為F之conjugate
23
4.The Discrete Fourier Transform


The DFT and inverse DFT really are inverse operations
Proof: Let A=F -1F. We want to show that A=I, where
1 n 1 ki kj
A[i, j ]    
n k 0

If i=j, then
1 n 1 ki ki 1 n 1 0 1
A[i, i ]       n  1
n k 0
n k 0
n

If i and j are different, then
1 n 1 ( j i ) k
A[i, j ]   
0
n k 0
(by Cancellati on Property)
24
4.The Discrete Fourier Transform
[a0,a1,a2,...,an-1]
[b0,b1,b2,...,bn-1]
Pad with n 0's
Pad with n 0's
[a0,a1,a2,...,an-1,0,0,...,0]
[b0,b1,b2,...,bn-1,0,0,...,0]
DFT
DFT
[y0,y1,y2,...,y2n-1]
[z0,z1,z2,...,z2n-1]
Convolution


The DFT and the
inverse DFT can be
used to multiply two
polynomials
So we can get the
coefficients of the
product polynomial
quickly if we can
compute the DFT (and
its inverse) quickly…
Component
Multiply
[y0z0,y1z1,...,y2n-1z2n-1]
inverse DFT
[c0,c1,c2,...,c2n-1]
(Convolution)
25
4.The Discrete Fourier Transform

Convolutions
127356
×
Convolutions
(3,11,37,47,42)
1
2
7
3
5
6
6 12 42
5 10 35
3 6 21
3 11 37 47 42
26
4.The Discrete Fourier Transform

Convolutions 127356
×
1
3
2
5
7
6
×
1
3
2
5
7
6
6 12 42
5 10 35
3 6 21
6 12 42
5 10 35
3 6 21
3 11 37 47 42
3 11 37 47 42
+
-
3 11
37
50
53
acyclic convolutions
37
(37,50,53)
3
11
44
31
negacyclic convolutions (37,44,31)
27
4.The Discrete Fourier Transform

Convolution theorem
CyclicConvolution(X, Y) = IDFT(DFT(X) · DFT(Y))
28
5.The Fast Fourier Transform


The FFT is an efficient algorithm for computing the DFT
The FFT is based on the divide-and-conquer paradigm:

If n is even, we can divide a polynomial
into two polynomials
and we can write
29
5.The Fast Fourier Transform
Given coefficients (a0,a1,a2,…,a7) and w primitive 8-th root of unity

1

1
1

1
M
1

1
1

1

1 1 1

2
4
1  
1  4  8

1  6  12
1
1


1
2

1
3

4
1
1


5
6
2 4 6
 8  10  12
3 6 9
 12  15  18
 4  8  12
 16  20  24
 5  10  15  20  25  30
 6  12  18  24  30  36
 7  14  21  28  35  42
1
1
 a 0   1


 6  a 2     3  5
 
12 
  a 4    2  6  10
 
 18  44 a6   3  9  15
1

a 0 
7 
  a 
 1
 14  a 2 
 
 21  a3 

 28  a 4 
  
 35  a5 
 
 42  a 6 
a 
 49  88  7 
1

1
1

1

1

1
1

1

1
 a1  1 1 1


 7  a3  1  2  4
 
14 
  a5  1  4  8
 
 21  44 a7  1  6  12
1
1
1
1
1
1
1
2 4 6

3
5
 4  8  12
 2  6  10
 6  12  18
 3  9  15
 8  16  24
 4  12  20
 10  20  30  5  15  25
 12  24  36  6  18  30
 14  28  42  7  21  35
 a 0   1


 6   a 2  0
 
 12  a 4  0
  
 18  44 a6  0
1

 a 0 
 7  a 
 2
14 
  a 4 
 
 21  a 6 

 28  a1 
  
 35  a3 
 
 42  a5 
a 
 49  88  7 
1
0 0 0  1 1 1

 0 0  1  2  4
0  2 0  1  4  8

0 0  3  1  6  12
 a1 

 6   a3 
 
12 
   a5 
 
 18  44 a7 
1
30
5.The Fast Fourier Transform
1 1 1

2
4
1  
1  4  8

1  6  12
1 1 1

2
4
1  
1  4  8

1  6  12
1
1
 a 0   1


 6  a 2     3  5
 
12 
  a 4    2  6  10
 
 18  44 a6   3  9  15
1  a0   1 0 0 0  1



 6  a 2  0  0 0  1
 
2
12 
  a 4  0 0  0  1

  
3 
18 
a
  44  6  0 0 0   1
1
 a1 

 7  a3 
 
14 
  a5 
 
21 
  44 a7 
1 1
1  a1 

 2  4  6  a3 
 
4
8
12 
    a5 
 
 6  12  18  44 a7 
1
31
5.The Fast Fourier Transform
1

1
  4  1, 
1
1

 8  16  24
1 1

2
1 
1  4

1  6
1
1
 4  12  20
 10  20  30  5  15  25
 12  24  36  6  18  30
 14  28  42  7  21  35
4 6
 8  12
 12  18
a 0 
a 
 2
28 

a 4 

 
 35  a 6 


 42  a1 
 49  88 a3 
a 
 5
a 7 
1
1
 a 0   1
   
3
5
 a 2    
 a     2  6  10
  4 
 a6   3  9  15
44
 a1 

 7  a3 
 
 14  a5 
 
21 
  44 a7 
1
32
5.The Fast Fourier Transform
令
M n/2
1
1 1

2
4
1  

1  4 8

1  6  12
1
6
 12
 18

a 0   1

 a  0

 2  
M
n
/
2

 a 4  0

  

a 6  0
M

a 0   1

 a  0
M  2   
 n / 2  a 4  0

  

a 6  0






0 0 0 
a1  
a  
 0 0 
 3 
M
n
/
2
a5  
0 2 0 

 
3
0 0  
a 7  

0 0 0 
a1  
a  
 0 0 
 3 
M
n/2
a5  
0 2 0 

 
3
0 0  
a 7  
33
5.The Fast Fourier Transform
a0 a1 a2 a3 a4
a5
a6 a7
34
5.The Fast Fourier Transform
T(n)=2T(n/2)+O(n)
The running time is O(n log n). [inverse FFT is similar]
35
Polynomial multiplication
Polynomial Multiplication
(a0,a1,a2,…,an-1)
(b0,b1,b2,…,bn-1)
p(x)=a0+a1x+…+an-1xn-1
q(x)=b0+b1x+…+bn-1xn-1
選w primitive m-th root of unity
Evaluation
FFT
Polynomial Multiplication
(m2n)
(p(w0), p(w1),…, p(wm-1) )
(q(w0), q(w1),…, q(wm-1) )
multiplication
(p(w0)*q(w0), p(w1)*q(w1),…
, p(wm-1)*q(wm-1) )
(c0,c1,c2,…,cn,…,cm-1)
The running time is
O(n log n).
Interpolation
IFFT
36
6.Schönhage-Strassen’s method
給定兩大數M,N。計算M*N
(1) Splitting (將整數轉成多項式)
令B=2k,則可產生degree 最多n-1之兩個多項式
p(x)=a0+a1B+…+an-1Bn-1
P=(a0,a1,a2,…,an-1)
q(x)=b0+b1B+…+bn-1Bn-1
Q=(b0,b1,b2,…,bn-1)
(2) Evaluation (找出多項式對應的點)
選w primitive m-th root of unity
(m2n)
FFT(P ,  )  ( p(1), p( ), , p( m 1 ))
FFT(Q,  )  (q(1), q( ), , q( m 1 ))
37
6. Schönhage-Strassen’s method
(3) Pointwise multiplication (點値相乘)
(C (1)  p(1)q(1), C ( )  p( )q( ), , C ( m1 )  p( m1 )q( m1 ))
(4) Interpolation (由點找出多項式)
對上述點執行the inverse Fourier transform
C(x)  IFFT( (C (1)  p(1)q(1), C ( )  p( )q( ), , C ( n1 )  p( m1 )q( m1 )),  )
 c0  c1 x    c 2 n 1 x 2 n2
38
6. Schönhage-Strassen’s method
(5)Recomposition (將多項式轉成整數)
C(x)  c0  c1 x    c2n2 x 2n2
令x=B
M * N  c0  c1B    c2n2 B
2 n 2
39
6. Schönhage-Strassen’s method
Complexity 分析
The running time is O(n log n) 單位?
The bound on the numerical errors α on the ci after the
FFT process can be proved to be
α ≤ 6n2B2log(n)ε
where ε ≅ 1.e-16
the number of digits of these basic
numbers should be of the order of log(B)+log(n)
40
6. Schönhage-Strassen’s method
Complexity 分析
The running time is O(n log n(log(B) + log(n))2)
然而如果針對所有乘法都採用recursive觀念，即
B1  N , B2  B1 ,
得到
T ( N )  N T ( N )  O(n log n)
T ( N )  O(n log n log log n)
41
7. Conclusion
Method*
classical
complexity
integer
O(n 2 )
1.58
Karatsuba 1962
O(n
Toom–3 1963
O(n1.46 )
)
Toom–Cook (knuth)
O(n2 2 lg n log n)
1966
SchönhageO(n log n log log n)
Strassen 1971
O(log* n)
Fürer 2007
O(n log n2
)
2
215
2
217
10,000位數
40,000位數
42