Download Lecture 8, practical part

FFT Based Multiplication
Reduction using Huffman Trees
Aspects of Computer Algebra
Eighth Lecture: Practical Part
Felix Fontein
April 23 and 24, 2013
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Overview
1
FFT Based Multiplication
Arithmetic in R[X ]/hX n + 1i
Fast Fourier Transform over D = R[X ]/hX n + 1i
Polynomial Multiplication using Fast Fourier Transform
The Wrapper
Timing
2
Reduction using Huffman Trees
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Overview
1
FFT Based Multiplication
Arithmetic in R[X ]/hX n + 1i
Fast Fourier Transform over D = R[X ]/hX n + 1i
Polynomial Multiplication using Fast Fourier Transform
The Wrapper
Timing
2
Reduction using Huffman Trees
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Overview
1
FFT Based Multiplication
Arithmetic in R[X ]/hX n + 1i
Fast Fourier Transform over D = R[X ]/hX n + 1i
Polynomial Multiplication using Fast Fourier Transform
The Wrapper
Timing
2
Reduction using Huffman Trees
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Arithmetic in R[X ]/hX n + 1i, 1/3
We represent an element of D := R[X ]/hX n + 1i as
(f0 , . . . , fn−1 ) ←→
n−1
X
fi X i + hX n + 1i.
i=0
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Arithmetic in R[X ]/hX n + 1i, 1/3
We represent an element of D := R[X ]/hX n + 1i as
(f0 , . . . , fn−1 ) ←→
n−1
X
fi X i + hX n + 1i.
i=0
Addition and subtraction in D: addition/subtraction of vectors
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Arithmetic in R[X ]/hX n + 1i, 1/3
We represent an element of D := R[X ]/hX n + 1i as
(f0 , . . . , fn−1 ) ←→
n−1
X
fi X i + hX n + 1i.
i=0
Addition and subtraction in D: addition/subtraction of vectors
Multiplication with ω i = X i + hX n + 1i:
Given α = aj X j + hX n + 1i, we have αω i = aj X i+j + hX n + 1i
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Arithmetic in R[X ]/hX n + 1i, 1/3
We represent an element of D := R[X ]/hX n + 1i as
(f0 , . . . , fn−1 ) ←→
n−1
X
fi X i + hX n + 1i.
i=0
Addition and subtraction in D: addition/subtraction of vectors
Multiplication with ω i = X i + hX n + 1i:
Given α = aj X j + hX n + 1i, we have αω i = aj X i+j + hX n + 1i
Write i + j = qn + r , 0 ≤ r < n
Then
X i+j + hX n + 1i = (X n )q X r + hX n + 1i = (−1)q X r + hX n + 1i
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Arithmetic in R[X ]/hX n + 1i, 1/3
We represent an element of D := R[X ]/hX n + 1i as
(f0 , . . . , fn−1 ) ←→
n−1
X
fi X i + hX n + 1i.
i=0
Addition and subtraction in D: addition/subtraction of vectors
Multiplication with ω i = X i + hX n + 1i:
Given α = aj X j + hX n + 1i, we have αω i = aj X i+j + hX n + 1i
Write i + j = qn + r , 0 ≤ r < n
Then
X i+j + hX n + 1i = (X n )q X r + hX n + 1i = (−1)q X r + hX n + 1i
Therefore
αω i = (−1)b(i+j)/nc aj X i+j mod n + hX n + 1i
Multiplying (f0 , . . . , fn−1 ) with ω i ⇒ negacyclic shift with offset i!
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Arithmetic in R[X ]/hX n + 1i, 2/3
(aj X j +hX n +1i)·(X shift +hX n +1i) = (−1)b(shift+j)/nc aj X shift+j mod n +hX n +1i
1
2
3
4
5
6
7
8
9
10
11
12
def negacyclicShift (R , shift , vector ) :
""" Computes a negacyclic shift of the vector over
the ring R . This corresponds to multiplying
with X shift in R[X ]/hX len(vector ) + 1i. """
result = [ None ] * len ( vector )
for i in xrange ( len ( result ) ) :
s , j = divmod ( i - shift , len ( result ) )
if ( s & 1) == 0: # s is even
result [ i ] = vector [ j ]
else :
result [ i ] = - vector [ j ]
return result
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Arithmetic in R[X ]/hX n + 1i, 3/3
13
14
15
16
17
18
def addVectors ( v1 , v2 ) :
" Adds ␣ the ␣ two ␣ given ␣ vectors . ␣ Assume ␣ they ␣ have ␣ the ␣
same ␣ length . "
result = [ None ] * len ( v1 )
for i in xrange ( len ( v1 ) ) :
result [ i ] = v1 [ i ] + v2 [ i ]
return result
19
20
21
22
23
24
25
def subVectors ( v1 , v2 ) :
" Subtracts ␣ the ␣ two ␣ given ␣ vectors . ␣ Assume ␣ they ␣
have ␣ the ␣ same ␣ length . "
result = [ None ] * len ( v1 )
for i in xrange ( len ( v1 ) ) :
result [ i ] = v1 [ i ] - v2 [ i ]
return result
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Overview
1
FFT Based Multiplication
Arithmetic in R[X ]/hX n + 1i
Fast Fourier Transform over D = R[X ]/hX n + 1i
Polynomial Multiplication using Fast Fourier Transform
The Wrapper
Timing
2
Reduction using Huffman Trees
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Fast Fourier Transform over D = R[X ]/hX n + 1i, 1/2
Algorithm 3.14: The Fast Fourier Transformation
Input: n = 2m , an n-th primitive root of unity ω with its
powers ω 0 , . . . , ω n−1 , and a vector a = (a0 , . . . , an−1 ) ∈ R n
Output: DFTω (a)
1
If n = 1 return (a0 );
2
Compute r0 = (a0 + an/2 , a1 + an/2+1 , . . . , an/2−1 + an−1 );
3
Compute
r1∗ = ((a0 − an/2 )ω 0 , (a1 − an/2+1 )ω 1 , . . . , (an/2−1 − an−1 )ω n/2−1 );
4
Compute (c0 , . . . , cn/2−1 ) = DFTω2 (r0 ) by recursion;
5
Compute (d0 , . . . , dn/2−1 ) = DFTω2 (r1∗ ) by recursion;
6
Return (c0 , d0 , c1 , d1 , . . . , cn/2−1 , dn/2−1 ).
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Fast Fourier Transform over D = R[X ]/hX n + 1i, 2/2
m
Given: vector f ∈ D 2
Therefore: elements of f are vectors which represent elements of D
26
27
28
29
30
31
32
33
34
35
36
37
38
39
def F F T _ U s i n g _ N e g a c y c l i c _ S h i f t s (R , m , shift , f ) :
if m <= 0: return f
k = (1 << ( m - 1) )
r0 = [ None ] * k ; r1 = [ None ] * k
for i in xrange ( k ) :
r0 [ i ] = addVectors ( f [ i ] , f [ i + k ])
r1 [ i ] = negacyclicShift (R , shift * i ,
subVectors ( f [ i ] , f [ i + k ]) )
c = F F T _ U s i n g _ N e g a c y c l i c _ S h i f t s (R , m -1 , shift *2 , r0 )
d = F F T _ U s i n g _ N e g a c y c l i c _ S h i f t s (R , m -1 , shift *2 , r1 )
result = [ None ] * ( k << 1)
for i in xrange ( k ) :
result [2 * i ] = c [ i ]
result [2 * i + 1] = d [ i ]
return result
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Overview
1
FFT Based Multiplication
Arithmetic in R[X ]/hX n + 1i
Fast Fourier Transform over D = R[X ]/hX n + 1i
Polynomial Multiplication using Fast Fourier Transform
The Wrapper
Timing
2
Reduction using Huffman Trees
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Polynomial Multiplication using Fast Fourier Transform,
1/6
40
P o l y M u l _ _ F F T _ F a l l b a c k _ B o u n d = 5 # must be at least 2
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
def FFTMultiplyImpl (R , m , f , g ) :
""" Computes the polynomial product of f and g
m
modulo X 2 + 1, where f and g are given as
vectors of length 2m . Assumes that 2
is invertible in R . """
n = 1 << m
global P o l y M u l _ _ F F T _ F a l l b a c k _ B o u n d
if m <= P o l y M u l _ _ F F T _ F a l l b a c k _ B o u n d :
res = Polynomial ( R ) . _Poly nomi al__ multi ply (f , g )
if len ( res ) < n :
res . extend ([ R . zero ] * ( n - len ( res ) ) )
else :
# assume len ( res ) < 2 * n
for i in xrange (n , len ( res ) ) :
res [ i % n ] = res [ i % n ] - res [ i ]
return res [: n ]
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Polynomial Multiplication using Fast Fourier Transform,
2/6
2
Set k := 2bm/2c and ` := 2dm/2e ;
3
Find f0 , . . . , f`−1 , g0 , . . . , g`−1 ∈ R[X ]<k such that
f =
`−1
X
fi X ki
i=0
4
5
and g =
`−1
X
gi X ki ;
i=0
Let D = R[X ]/hX 2k + 1i and let ω2 = X 2 + hX 2k + 1i if k = ` or
ω2 = X + hX 2k + 1i if k < ` (now ω2 is a primitive (2`)-th root of
unity);
Let
f∗ =
`−1
X
(fi + hX 2k + 1i)ω2i Y i ∈ D[Y ]<`
i=0
and g ∗ =
`−1
X
(gi + hX 2k + 1i)ω2i Y i ∈ D[Y ]<` ;
i=0
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Polynomial Multiplication using Fast Fourier Transform,
3/6
58
59
60
61
k = 1
ell =
omega
# Now
of
<< ( m /2)
1 << (( m +1) /2)
= 1 + int ( k == ell )
X omega + hX 2k + 1i is a primitive 2* ell - th root
unity in D = R[X ]/hX 2k + 1i.
62
63
64
65
66
fstar = [ None ] * ell
for i in xrange ( ell ) :
fstar [ i ] = negacyclicShift (R , i * omega , f [ k * i :
k *( i +1) ] + [ R . zero ] * k )
P`−1
P`−1
# f ∗ = i=0 (fi + hX 2k + 1i)ω2i Y i for f = i=0 fi X ki
67
68
69
70
71
gstar = [ None ] * ell
for i in xrange ( ell ) :
gstar [ i ] = negacyclicShift (R , i * omega , g [ k * i :
k *( i +1) ] + [ R . zero ] * k )
P`−1
P`−1
# g ∗ = i=0 (gi + hX 2k + 1i)ω2i Y i for g = i=0 gi X ki
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Polynomial Multiplication using Fast Fourier Transform,
4/6
6
Compute f 0 = DFTω22 (f ∗ ) using Algorithm 3.14;
7
Compute g 0 = DFTω22 (g ∗ ) using Algorithm 3.14;
8
9
Compute h0 as the component-wise product f 0 · g 0 (as elements of
D ` );
Compute h∗ = DFTω2`−2 (h0 ) using Algorithm 3.14;
2
72
73
74
75
76
77
fprime = F F T _ U s i n g _ N e g a c y c l i c _ S h i f t s (R , ( m +1) /2 ,
2* omega , fstar )
gprime = F F T _ U s i n g _ N e g a c y c l i c _ S h i f t s (R , ( m +1) /2 ,
2* omega , gstar )
hprime = [ None ] * ell
for i in xrange ( ell ) :
hprime [ i ] = FFTMultiplyImpl (R , m /2+1 ,
fprime [ i ] , gprime [ i ])
hstar = F F T _ U s i n g _ N e g a c y c l i c _ S h i f t s (R , ( m +1) /2 ,
-2* omega , hprime )
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Polynomial Multiplication using Fast Fourier Transform,
5/6
10
78
79
80
81
82
83
84
85
86
87
88
∗
), compute and return
If h∗ = (h0∗ , . . . , h`−1
P`−1 −1 ∗ 2`−i ki
h = i=0 ` (hi ω2 )X modulo X n + 1, where hi∗ ω22`−i is first
evaluated in R[X ]/hX 2k + 1i and then interpreted as a polynomial in
R[X ]<2k .
h = [ R . zero ] * n
for i in xrange ( ell ) :
hc = negacyclicShift (R , -i * omega , hstar [ i ])
pos = i * k
if i == ell - 1:
# Last coeff . has to be computed modulo X ˆ n +1
for j in xrange ( k ) : h [ pos + j ] = h [ pos + j ]+ hc [ j ]
for j in xrange ( k ) : h [ j ] = h [ j ] - hc [ k + j ]
else :
# All others are not affected by modulo X ˆ n +1
for j in xrange (2* k ) : h [ pos + j ]= h [ pos + j ]+ hc [ j ]
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Polynomial Multiplication using Fast Fourier Transform,
6/6
10
89
90
91
92
93
94
∗
), compute and return
If h∗ = (h0∗ , . . . , h`−1
P`−1 −1 ∗ 2`−i ki
h = i=0 ` (hi ω2 )X modulo X n + 1, where hi∗ ω22`−i is first
evaluated in R[X ]/hX 2k + 1i and then interpreted as a polynomial in
R[X ]<2k .
# Scale with 1/ ell in R
ell_inv = R . invert ( R . coerceInt ( ell ) )
assert ell_inv is not None
for i in xrange ( n ) :
h [ i ] = h [ i ] * ell_inv
return h
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Overview
1
FFT Based Multiplication
Arithmetic in R[X ]/hX n + 1i
Fast Fourier Transform over D = R[X ]/hX n + 1i
Polynomial Multiplication using Fast Fourier Transform
The Wrapper
Timing
2
Reduction using Huffman Trees
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
The Wrapper for Polynomial Multiplication, 1/2
95
96
97
def IsPowerOfTwo ( n ) :
" Returns ␣ true ␣ if ␣ the ␣ integer ␣ n ␣ is ␣ a ␣ power ␣ of ␣ two . "
return ( n & ( n - 1) ) == 0
98
99
100
101
102
103
104
105
106
107
108
109
110
def FFTMultiply (f , g ) :
# Compute smallest power of two , n , which
satisfies n > f . deg () + g . deg ()
n = f . deg () + g . deg () + 1
m = bits . floor_log2 ( n )
if not IsPowerOfTwo ( n ) :
m += 1
# Create coefficient vectors of length n
R = f.R
fvec = f . p + [ R . zero ] * ( n - len ( f . p ) )
gvec = g . p + [ R . zero ] * ( n - len ( g . p ) )
# Multiply
resvec = FFTMultiplyImpl (R , m , fvec , gvec )
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
The Wrapper for Polynomial Multiplication, 2/2
111
112
113
114
115
116
# Remove coefficients which must be zero
resvec = resvec [: f . deg () + g . deg () + 1]
# Create polynomial out of result
res = polynomials . Polynomial (R , None , resvec )
res . _ P o l y n o m i a l __n or ma liz e ()
return res
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Overview
1
FFT Based Multiplication
Arithmetic in R[X ]/hX n + 1i
Fast Fourier Transform over D = R[X ]/hX n + 1i
Polynomial Multiplication using Fast Fourier Transform
The Wrapper
Timing
2
Reduction using Huffman Trees
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Timing 1/2
Degree of f compared to total degree: 50%
31.62
17.78
10.00
FFT(2)
FFT(3)
FFT(4)
FFT(5)
FFT(6)
FFT(7)
FFT(8)
FFT(9)
FFT(10)
FFT(11)
Karatsuba
Classic
5.62
3.16
1.78
1.00
0.56
0.32
0.18
0.10
0.06
0.03
0.02
0.01
16
32
64
128
x -axis: total degree + 1
256
512
1024
2048
4096
8192
16384
y -axis: time in seconds (logarithmic scale)
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Timing 2/2
Degree of f compared to total degree: 75%
31.62
17.78
10.00
FFT(2)
FFT(3)
FFT(4)
FFT(5)
FFT(6)
FFT(7)
FFT(8)
FFT(9)
FFT(10)
FFT(11)
Karatsuba
Classic
5.62
3.16
1.78
1.00
0.56
0.32
0.18
0.10
0.06
0.03
0.02
0.01
16
32
64
128
x -axis: total degree + 1
256
512
1024
2048
4096
8192
16384
y -axis: time in seconds (logarithmic scale)
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Overview
1
FFT Based Multiplication
Arithmetic in R[X ]/hX n + 1i
Fast Fourier Transform over D = R[X ]/hX n + 1i
Polynomial Multiplication using Fast Fourier Transform
The Wrapper
Timing
2
Reduction using Huffman Trees
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Reduction using Huffman Trees 1/5
1
2
3
class Node ( object ) :
left = None
right = None
4
5
6
7
8
9
10
11
def c r e a t e H u f f m a n T r e e _ T u p l e L i s t ( data ) :
""" Creates a Huffman tree with the given data at
its leaves . Each element in data should be a
tuple (v , d ) , where v will the the value of
the leaf and d assigned to its data
attribute . """
...
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Reduction using Huffman Trees 2/5
12
13
14
15
16
17
18
def walkTreeNLR ( root , function ) :
""" Walks tree in order node - > left - > right , and
calls the given function for every node . """
if root is None : return
function ( root )
walkTreeNLR ( root . left , function )
walkTreeNLR ( root . right , function )
19
20
21
22
23
24
25
26
def walkTreeLRN ( root , function ) :
""" Walks tree in order left - > right - > node , and
calls the given function for every node . """
if root is None : return
walkTreeLRN ( root . left , function )
walkTreeLRN ( root . right , function )
function ( root )
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Reduction using Huffman Trees 3/5
1
2
3
4
5
6
7
8
9
10
11
def Reduction_Huffman ( value , moduli ) :
""" Computes value modulo moduli [ i ] for all i .
Uses Huffman trees . """
if len ( moduli ) == 0:
return []
# Create Huffman tree
results = [ None ] * len ( moduli )
valmodList = [ None ] * len ( moduli )
for i in xrange ( len ( moduli ) ) :
valmodList [ i ] = moduli [ i ]. deg () , ( moduli [ i ] ,
results , i )
root = c r e a t e H u f f m a n T r e e _ T u p l e L i s t ( valmodList )
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Reduction using Huffman Trees 4/5
12
13
1
2
3
4
5
6
7
8
9
10
# Compute products
walkTreeLRN ( root , _ _ s p l i t A n d C o m p u t e P r o d u c t s 2 )
def _ _ s p l i t A n d C o m p u t e P r o d u c t s 2 ( node ) :
""" Helper function for computing the product tree
( for Huffman trees ) ; compare Algorithm 5.6. """
if node . is_leaf :
node . prod = node . data [0]
node . resultlist = node . data [1]
node . index = node . data [2]
del node . data
else :
node . prod = node . left . prod * node . right . prod
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part
FFT Based Multiplication
Reduction using Huffman Trees
Reduction using Huffman Trees 5/5
14
15
16
17
1
2
3
4
5
6
7
8
9
# Compute reductions
root . value = value % root . prod
walkTreeNLR ( root , _ _ si m ul ta n eo u sR ed u ct i on )
return results
def _ _s i m u l t a n e o u s Re d uc t io n ( node ) :
""" Helper function for computing simultaneous
reductions ( for Huffman trees ) ; compare
Algorithm 5.7. """
if node . is_leaf :
node . resultlist [ node . index ] = node . value
else :
node . left . value = node . value % node . left . prod
node . right . value = node . value % node . right . prod
Felix Fontein
Aspects of Computer Algebra Eighth Lecture: Practical Part