Download Slides

Miroslav Morháč - Institute of Physics, Slovak Academy of Sciences, Bratislava,
Slovakia
Error-free algorithms to solve special and general discrete
systems of linear equations.
ACAT2007, April 23-27, Amsterdam, Netherlands
1. Introduction
•the solution of linear algebraic equations is a very frequent task in numerical mathematics and
experimental physics at all.
•when solving such a set of linear equations one often meets the problem of ill-conditioned matrix of
the set.
•for large dense sets of linear equations, which are as a rule ill-conditioned, the stability of the
solution cannot be guaranteed.
•rounding and truncation errors, during the numerical computations, involved in obtaining the
solution to the problem cannot be tolerated.
Illustrative example
Let us have ill-conditioned set of two linear equations
x + 3y = 4
x + 3.00001y = 4.00001
(1)
which has the solution x = 1, y = 1.
On the other hand let us consider the set
x + 3y = 4
x + 2.99999y = 4.00002
(2)
This set has the solution x = 10, y = -2. Very small change in the coefficients (0.00002 and 0.00001)
caused enormous changes in the solution. The inverse matrix of (1) contains elements of the order
of
It demonstrates
its ill-conditionality.
105.
•we are motivated by the fact that in scientific computations there are large classes of
ill-conditioned problems and there are also numerically unstable algorithms
•digital computer is a finite machine
•it is capable of representing internally only a finite set of numbers
•there exist difficulties associated with the attempts at approximating arithmetic in the
field of real numbers (R,+,-) by using the finite set of so-called floating-point numbers F,
more appropriately called the set of computer-representable numbers
•there is no possibility of representing the continuum of real numbers in any detail.
•a “practical” solution is to represent a real number by the closest computerrepresentable number, thereby introducing the rounding error
•it is well known that computers can perform certain arithmetic operations exactly if the
operands are integers
•moreover there are many situations in physics when we work with integer input data.
•processing of spectra (histograms) can serve as a good example
•when solving ill-conditioned problems during the analysis of spectra it is not reasonable
to leave the world of exact integers and thus introduce instability
•it motivates us to consider integer arithmetic as a mean of avoiding rounding errors in
the hope that certain ill-conditioned problems can be solved exactly
•residue number system is an example of a number system with which we can do exact
arithmetic
•the aim of the contribution is to present error-free algorithms to solve ill-conditioned
systems of linear equations
The question is how can we proceed when the solution of ill-conditioned linear system of equations
can be expressed as rational fractions of integers
•we can increase the precision of floating point representation – it does not need to guarantee the
correct result
•we can work with long integers, it is many times very cumbersome
•we can work with integers in finite rings using residue class or modulo arithmetic and at the end of
calculation to convert the modulo representation into real numbers
Examples of associations in modulo arithmetic (M=17)
Negative numbers
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
0
1
2
3
4
5
6
7
8
-8
-7
-6
-5
-4
-3
-2
-1
Inverse numbers
01
11
21
31
41
51
61
7 1
81

1
9
6
13
7
3
5
15
91 101 111 121 131 141 151 161
2
12
14
10
4
11
8
16
Rational fractions
D
R
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
1
10
1
11
1
12
1
13
1
14
1
15
1
16
1
2
2
4
2
6
2
8
2
10
2
12
2
14
2
16
2
1
2
3
2
5
2
7
2
9
2
11
2
13
2
15
2
3
3
6
3
9
3
12
3
15
3
1
3
4
3
7
3
10
3
13
3
16
3
2
3
5
3
8
3
11
3
14
3
4
4
8
4
12
4
16
4
3
4
7
4
11
4
15
4
2
4
6
4
10
4
14
4
1
4
5
4
9
4
13
4
5
5
10
5
15
5
3
5
8
5
13
5
1
5
6
5
11
5
16
5
4
5
9
5
14
5
2
5
7
5
12
5
6
6
12
6
1
6
7
6
13
6
2
6
8
6
14
6
3
6
9
6
15
6
4
6
10
6
16
6
5
6
11
6
7
7
14
7
4
7
11
7
1
7
8
7
15
7
5
7
12
7
2
7
9
7
16
7
6
7
13
7
3
7
10
7
8
8
16
8
7
8
15
8
6
8
14
8
5
8
13
8
4
8
12
8
3
8
11
8
2
8
10
8
1
8
9
8
9
9
1
9
10
9
2
9
11
9
3
9
12
9
4
9
13
9
5
9
14
9
6
9
15
9
7
9
16
9
8
9
10
10
3
10
13
10
6
10
16
10
9
10
2
10
12
10
5
10
15
10
8
10
1
10
11
10
4
10
14
10
7
10
11
11
5
11
16
11
10
11
4
11
15
11
9
11
3
11
14
11
8
11
2
11
13
11
7
11
1
11
12
11
6
11
12
12
7
12
2
12
14
12
9
12
4
12
16
12
11
12
6
12
1
12
13
12
8
12
3
12
15
12
10
12
5
12
13
13
9
13
5
13
1
13
14
13
10
13
6
13
2
13
15
13
11
13
7
13
3
13
16
13
12
13
8
13
4
13
14
14
11
14
8
14
5
14
2
14
16
14
13
14
10
14
7
14
4
14
1
14
15
14
12
14
9
14
6
14
3
14
15
15
13
15
11
15
9
15
7
15
5
15
3
15
1
15
16
15
14
15
12
15
10
15
8
15
6
15
4
15
2
15
16
16
15
16
14
16
13
16
12
16
11
16
10
16
9
16
8
16
7
16
6
16
5
16
4
16
3
16
2
16
1
16
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
2. Convolution systems
•in many cases the dynamic behavior of a linear system can be described by means of impulse
response function h(n).
•the periodical convolution of finite sets x(n) and h(n) is
N 1
y (n)   h(n  m) x(m), n  0,1,
(3)
, N 1
m0
•very frequently the advantageous property of factorization of the convolution of two signals to the
product of their Fourier coefficients is utilized [1], [2].
Fi  y(n)  Fi h(n)  Fi  x(n) , i  0,1,
, N 1
(4)
•since the Fourier transform cannot be computed with absolute precision, the number theoretical
transforms were defined with the aim to carry out the fast error-free convolution of data.
•these transforms are defined in the ring of integers using the operations carried out in modulo M
arithmetic, where M is prime
•direct and inverse Number Theoretical Transforms (NNT) can be defined
 N 1

X (k )   x(n) kn  (mod M ) k  0,1, , N  1
 n 0

 1 N 1

x(n)   N  X (k )  kn  (mod M ) n  0,1, , N  1
k 0


(5)
(6)
where
a. N  N 1 (mod M )  1
b.  N (mod M )  1
M is prime and N must divide M-1.
•Mersenne Transforms (Mersenne numbers M  2q  1,q is prime, some Mersenne numbers are
primes
t
•Fermat Transforms (Fermat numbers M  Ft  22  1, F0  3, F1  5, F2  17, F3  257, F4  65537)
•Fermat transform is the most suitable, fast transform algorithm exists. The first 5 Fermat numbers
are primes
•there are many applications for error-free convolution calculations (filtration).
•however, the problem of precision is much more critical in the inverse operation - deconvolution
2.1 Precise deconvolution using the Fermat number transform [3]
Convolution system (3) can be written as
 y  0
 h  0 ,
h  N  1 ,

 
h  0 ,
 y 1
  h 1 ,
 M
 M
M

 
 y  N  2    h  N  2  , h  N  3 ,

 
 y  N  1   h  N  1 , h  N  2  ,
or
y  H x
L
L
L
L
h  2 ,
h  3 ,
M
h 0 ,
h 1 ,

h 1
  x 0

 x 1


h  2




 g M
M



h  N  1   x  N  2 

 
h 0
  x  N  1 
(7)
(8)
The solution of any regular system of linear equations (8) can be expressed as
x
1
 M 0 x0  M 1x1  M 2 x2 L  M m x m 
D
(9)
where M is chosen modulus (Fermat number) and m is a finite number
Substituting this expression into matrix equation (8) we obtain
1
 M 0  H  x0  M 1  H  x1  L  M m  H  xm 
D
(10)
 1 1


L
y

D

H

x

H

x

L
H

x


0
1
m  0
 M M




(11)
y
or
1
M
Where D is determinant of the matrix H. Now we can iteratively calculate
y1 
1
 y  D  H  x0  ,
M
y j 1 
1
y j  H  x j  ,
M
j  1, 2,K ,m.
(12)
If the vector y j 1 equals the zero vector the calculation can be finished.
To calculate modulo solution of (8) we can utilize the Fermat number transform. For transformed
values it holds
Ri  x   Ri 1 h   Ri  y  (mod M ) i  0,1,K ,N 1
(13)
Using inverse Fermat transform we obtain modulo M solution of vector
x(mod M )  xM 
D
D
xM (mod M )  M xM (mod M )
D
D
x
(14)
where
 N 1

D  k  M  DM , x0  DM  x M , DM   H  i   (mod M )
 i 0

(15)
Then
x0  x  D  M  x1  M  x 2 K  Mx m K 
(16)
Substituting (16) to (12) we get
y1  H  x1  M  H  x2  K M  H  xm 
y j (mod M )  H  x j
Here to find particular solution
to the next iteration step
y j 1 
or in general
(17)
x j we can apply again Fermat transform. Then we can proceed
yj  H xj
(18)
M
Final solution can be expressed
1 k
x( i )   x j ( i )  M j , i  0,1,K ,N  1
D j 0
(19)
Exact algorithm to calculate determinant of the convolution matrix [3], [4]
The determinant of the convolution matrix of the system (8) can be calculated as follows
1
D   1 2
 N 1 N  2 1
N 1

k
  h0  hW
 h2W 2k  L  hN 1W  N 1 k
1
k 0

(20)
 j  2  
where W  e N
and N is a power of 2. It follows from relation (20) that the determinant of such a
matrix· can be expressed as the product of the coefficients of the Fourier transform of the response vector
h . Since in the deconvolution algorithm using the Fermat transform the exact integer value of the
determinant is needed, the determinant cannot be calculated simply by multiplication of the Fourier
coefficients of the response. The Fourier transform is used only formally. For details we refer to [3].
Let us illustrate the algorithm by an example with N  8 . Applying DFT to vector h we obtain bit
reversed vector
1
1
1
1
1
1
1
1

1 W 2
1 W 2

2
1 W 2
1 W
1
W
W2
W3

W 2 W 3
1 W
1 W 3 W 2
W

1 W 3 W 2 W
1
1
1
1
1 W2
1 W 2
1 W
1
W
1 W 3
1
W3
1  h0 
 
1  h1 
1 W 2   h2 
  
1 W 2   h3 


W 2 W 3   h4 
  
W 2
W 3   h5 
W 2 W   h6 

2
W
W   h7 
1
1
 h0  h1  h2  h3  h4  h5  h6  h7



 h0  h1  h2  h3  h4  h5  h6  h7

 h0  h2  h4  h6    h1  h3  h5  h7  W



 h0  h2  h4  h6    h1  h3  h5  h7  W




2
3
 h0  h4    h1  h5  W   h2  h6  W   h3  h7  W 

2
3 
 h0  h4    h1  h5  W   h2  h6  W   h3  h7  W 
 h  h    h  h  W 3   h  h  W 2   h  h  W 
4
1
5
2
6
3
7
 0

3
2
 h  h    h  h  W   h  h  W   h  h  W 
4
1
5
2
6
3
7
 0

 a0
  X  1.

  0
b
 0
  X 1   2.
2
c  c W
 

1
 0
 X2  
3.
2
 c0  c1W
  X3  



2
3 

 d 0  d1W  d 2W  d 3W   X 4  
d  d W  d W 2  d W 3   X  
1
2
3
 0
  5 
 4.
 d 0  d1W 3  d 2W 2  d 3W 3   X 6  

 
3
2
3
X  
d

d
W

d
W

d
W


1
2
3
 0
  7 
The numbers in the first and the second group are real. In the third group we multiply the numbers
and in the fourth the numbers
group
group
group
group
X2 , X3
X 4 , X 5 , X 6 , X 7 • The fact that W 4  1 must be taken into account. Then
X 2  X 3   c0  c1W 2  c0  c1W 2   c02  c12
X 4  X 5   d 0  d1W  d 2W 2  d 3W 3  d 0  d1W  d 2W 2  d 3W 3  
  d 02  2d1 d 3  d 22    d12  2d 0 d 2  d 32 W 2  e0  e1W 2
X 6  X 7   d 0  d1W 3  d 2W 2  d 3W
 d
0
 d1W 3  d 2W 2  d 3W
  d 02  2d1 d 3  d 22    d12  2d 0 d 2  d 32 W 2  e0  e2W 2
X 4  X 5  X 6  X 7   e0  e1W 2  e0  e1W 2   e02  e12

Generally, let us have the vectors






k
Xn  






where
n
0
X n , 0 X n , 0 X n ,K , where for
X n  j  1






k 1
Xn  









M



k

Xn  j / 2

k
X n  j / 2  1 

M

k

X n 1

k

Xn  0 
k
is the number of reduction
n  0,1,K ,log 2 N  1;
negative transition to the topmost position
0
X n 1
X n and
k 1
X n there is
 k Xn  0 


k
X n  j  1


M

k
X n  j / 2  1 

k

Xn  j / 2 

M


k
Xn  2 

k

X n 1

k is the number of cyclic shifts with a
k  0,1,K , N / 2n 1 ;
j  N / 2n ;
components of the basic element of the group. Then for the reduced vector



k
 1

 


 1 0
 

k
0
X n1
N
is the number of
there holds




 k X 2  0   k X 2  j / 2   2  1i  k X  i  1  k X  j  i  1  

n
n
n
n


i 1

 

M

j
1



2
 0 X 2  0   0 X 2  j / 2   2  1i  0 X  i  1  0 X  j  i  1  

n
n
n
n


i 1

 
M
j
1
2
Using the described procedure we can calculate the exact integer value of the determinant of the convolution
matrix which is used in the deconvolution algorithm.
Illustrative example
Let us have the system
3
2

0

0
0
3
2
0
0
0
3
2
2   x  0   3 


0   x 1  5 


0   x  2   3 
  
 
3   x  3   0 


where the determinant D  65 . We shall use the modulus
N  4 . Then the transform matrices are as follows
1 1 1 1
1 4 16 13

T 
1 16 1 16 


1 13 16 4 
and T 1
M  17 , which is the Fermat number and
1 1 1 1
1 13 16 4 

 13  
1 16 1 16 


1 4 16 13
Then
1 1 1 1 3 
5 
1 4 16 13  2 
11 





ht  T  h  mod M  
mod M    

1 16 1 16   0 
1 

  
 
1 13 16 4   0 
12 
It follows that
ht 1  51 ,111 ,11 ,121 
T
 mod M   7 ,14,1,10
We calculate the vector of the transformed output values
T
1-st iteration step
Ym0  T  y m0
11 
1 1 1 1 3 
3 
1 4 16 13 5 






 mod 17  1 
1 16 1 16  3 
 

  
14 
1 13 16 4   0 
Then there holds
11
 3
X0  
 1

14

7
9 
8 

 14 
 mod 17  1 
 1
 

 10 
4
Applying the inverse transform we obtain
1 1 1 1 9 
14 
14 
1 13 16 4  8 
15 
 
     mod 17      D 15   mod 17  
x 0  13  
1 16 1 16  1 
8  D  8 

  
 
 
1 4 16 13  4 
6 
6 
9 
14 
14 
 
15 
15 
DM  
1 6 
14  

 mod 17      mod 17   10
65
D 8 
65 8
 
 
 
16 
6 
6 
where DM was calculated using relation (15). Let us suppose that the positive numbers are represented in
the range 0 , M  1 / 2 and the negative ones in the range  M  1 / 2  1,M  1 . Then we obtain the
negative values for numbers greater than  M  1 / 2 by subtracting the modulus. It follows that
 8 
 
1  6
x0 
65  7 
 
 1
Further we calculate
3

1 2

y 0  H  x0 
65  0

0
0 0 2   8
 26 
3 0 0   6  1  2 


,
2 3 0   7  65  9 
  


0 2 3   1
 17 
 195  26   13

  2   19
325
1
1



   
y1 
y m0  D  y 0 
M
17  195  9   12

 
  
  0   17    1


y m1  y1  mod M   13,19 ,12 ,1
T
 mod 17   13, 2,12,1
T
2-nd iteration step
Further in the next iteration we obtain
Ym1  T  y m1  mod M   11,5,5,14
T
X1  11  7 ,5 14 ,5 1,14 10
T
 mod 17   9, 2,5, 4
T
Using the inverse transform we calculate the vector x1
x1  T 1  X1  mod M   5,3, 2,16
T
 M 1 / 2
Adjusting the elements greater than
we obtain
x1  5,3, 2 , 1
T
Then we calculate
y1  H  x1  13,19,12,1
T
y2 
y1  y1 1 
T
T
T

13,19,12,1  13,19,12,1   0,0,0,0

M
17
The zero vector y 2 indicates the end of calculation. The resulting vector is then obtained by
1
1
T
T
 x0  x1 17   8,6, 7 , 1  17  5,3,2, 1  
D
65
1
T
  77 ,57 , 27 , 18
65
x
2.2 Multidimensional error-free deconvolution using the Fermat number transform
[5]
The periodical k-dimensional convolution of the finite sets
h  n1 , n2 ,..., nk  is defined by
y  n1 , n2 ,..., nk  
N 1
N 1
N 1
  ...  x  m , m ,...m   h  n  m , n
m1  0 m2  0
mk  0
1
2
k
1
1
2
x  n1 , n2 ,..., nk 
 m2 ,..., nk  mk 
Analogously to one-dimensional case in [5] we have derived the error-free algorithm of kdimensional deconvolution as well as algorithm to calculate the determinant of the k-dimensional
convolution matrix.
and
2.3 Error-free deconvolution using polynomial algebra concept [6], [7]
•polynomial algebra plays an important role in digital signal processing because convolutions can
be expressed in terms of operations on polynomials ([2]).
•polynomial algebra is the basis of one group of fast and error-free one- and multidimensional
convolution algorithms.
Now suppose the elements of h  m  ,x  l  in [3] to be coefficients of the polynomials H  z  and
X  z  of degree N  1 . Hence we have
N 1
H  z    h  m z m
(21)
m0
N 1
X  z    x l  zl
(22)
l 0
If we multiply H  z  by X  z  , the resulting polynomial will be of degree 2 N  2 . Thus
Y  z  H  z X  z 
2 N 2
 y n z
n
.
(24)
n 0
This means that the convolution of two sequences can be treated as the product of two polynomials.
If the one-dimensional convolution defined by (1) is cyclic the indices n, m, l, are calculated modulo
N. This implies that Z N  mod N   1 and therefore the cyclic convolution can be considered as the
multiplication of two polynomials
Y  z  H  z X  z
where the powers of the polynomial variable z are calculated modulo N or by
Y  z   H  z   X  z  mod  z N  1 .
(25)
Let us proceed to the multidimensional case of convolution. The k-dimensional cyclic convolution
of the discrete input signal x and the response signal h is defined as
y  n1 ,K nk  
where
N1 1
Nk 1
 ,K ,  h  m ,K m  x  n  m ,K ,n
m1  0
mk  0
1
k
1
1
k
 mk 
n1  0,N1 1 , n2  0,N2 1 , K , nk  0,Nk 1 and indices
(26)
n1  m1 are calculated
modulo N1 the indices n2  m2 are calculated modulo N 2 , etc. By considerations analogical with
those of the one-dimensional case the polynomial expression of the k-dimensional convolution is
obtained in the form
(27)
Y  z1 ,z2 ,K ,zk   H  z1 ,z2 ,K ,zk   X  z1 ,z2 ,K ,zk  ,
where the powers of the variable z1 are calculated modulo N1 and the powers of the variable zk are
calculated modulo N k .
In the case of the convolution of the signals h, x we can compute the one-, resp. the k-dimensional
output signal y using the relations (25), resp. (27). In the case of deconvolution, i.e. when we know
the output signal y and the response signal h, the input signal can be formally expressed in
polynomial form as
(28)
X  z   Y  z   H 1  z  ,
resp.
(29)
X  z1 ,z2 ,K ,zk   Y  z1 ,z2 ,K ,zk   H 1  z1 ,z2 ,K ,zk  ,
This implies that we have to execute operation of the inversion of the polynomials H  z  , resp.
H  z1 ,z2 ,K ,zk  .
Illustrative example for 1D deconvolution
Let the vectors of the output signal, resp. the impulse response be
y  3,1, 2 ,1 ,
T
resp.
h  1, 4, 2, 0 .
T
According to (5) there holds
3  z  2z
2
 z 3   1  4 z  2 z 2   X  z  .
From that we have
3  z  2z 2  z3
X z 
.
2
1  4z  2z
Multiplying the numerator and the denominator by the polynomial H   z  modulo z N  1 we
obtain
 3  z  2 z  z 1  4 z  2 z    3  9 z  4 z  5z  .
X  z 
1  4 z  2 z 1  4 z  2 z 
 5  12 z 
2
3
2
2
2
2
3
2
In the following sections of the paper the product H  z   H   z  will be called the reduction of the
polynomial H  z  with respect to the variable z . If we multiply the numerator and the denominator
by the polynomial 5  12z 2 , the resulting polynomial is
 3  9 z  4 z  5 z  5  12 z   63  105 z  56 z
X  z 
119
 5  12 z  5  12 z 
2
3
2
2
2
2
 133 z 3
.
Illustrative example for 2D deconvolution
Let the matrix of the two-dimensional output signal, resp. the two-dimensional impulse response be
 3, 2 
y
,
1, 4 
resp.
 2 , 3
h
.
1, 3
Using (27) for k = 2 we have
 3  z1    2  4 z1   z2    2  z1    3  3 z1   z2   X  z1 ,z2 
or
 3  z1    2  4 z1   z2 
X  z1 ,z2   
.
 2  z1    3  3z1   z2 
By multiplying the numerator and the denominator by the polynomial H  z1 ,  z2  i.e. by its reduction
with respect to the variable z 2 , we get
 3  z1    2  4 z1   z2   2  z1    3  3z1   z2  11  13z1    4  2 z1  z2
X  z1 ,z2   

13  14 z1
 2  z1    3  3z1   z2   2  z1    3  3z1   z2 
Now we multiply this result by the polynomial 13  14z1
11  13z1    4  2 z1  z2  13  14 z1   39  15 z1    24  30 z1  z2
X  z1 ,z2   

.
27
13  14 z1 13  14 z1 
The resulting matrix x is then
x
1  39, 24 
.
27  15, 30 
Let us assume that the dimensions of the convolution system N1 ,N 2 ,K ,N k to be powers of 2. Then
the algorithm of inversion of the polynomial H  z1 ,z2 ,K ,zk  and its simultaneous multiplication by
the polynomial Y  z1 ,z2 ,K ,zk  can be expressed as successive reductions of the response function
(autoconvolutions) and convolutions of Y  z1 ,z2 ,K ,zk  with reduced response (crossconvolutions).
The  p  1 th reduction of the response in the direction k is
hk  l1 ,l2 ,K ,lk 1 , 2
p 1
p 1
 lk  
N1 1 N2 1
 Nk 

 1
Nk 1 1  2 p 
 K  
i1  0
i2  0
ik 1  0
ik  0
p
hk  i1 ,i2 ,K ,ik 1 , 2 p  ik , 
 p hk  m1 ,m2 ,K ,mk 1 ,mk    1 k , (30)
i
where
m1  l1  i1  mod N1 ,
m2  l2  i2  mod N2 ,
mk 1
M
 lk 1  ik 1  mod Nk 1 ,
mk   2 p 1  lk  2 p  ik  mod N k ,
 N 
l1  0,N1  1 , l2  0,N 2  1 ,K ,lk 1  0,N k 1  1 , lk  0, p k1   1 ,
2 
and the number of the reduction p  0,1,K ,log 2 N k 1 . •
Moreover, let
0
nk i1 ,i2 ,K ,ik   y i1 ,i2 ,K ,ik  ,
where
i1  0,N1 1 , i2  0,N2 1 ,K ,ik  0,Nk 1 ,
Then the calculation of the
 p  1 th reduction of the signal
nk in the direction k is given by the
formula
p 1
nk  l1 ,l2 ,K ,lk 1 ,lk  
N1 1 N2 1
 Nk 

 1
Nk 1 1  2 p 
 K  
i1  0
i2  0
ik 1  0
ik  0
p
nk  i1 ,i2 ,K ,ik 1 , 2 p  ik , 
 p hk  m1 ,m2 ,K ,mk 1 ,mk    1 k
i
where
m1  l1  i1  mod N1 ,
m2  l2  i2  mod N2 ,
mk 1
M
 lk 1  ik 1  mod Nk 1 ,
mk   lk  2 p  ik  mod N k ,
l1  0,N1 1 , l2  0,N2 1 ,K ,lk 1  0,Nk 1 1 , lk  0,Nk 1
and the number of the reduction p  0,1,K ,log 2 N k 1 .
(31)
We repeat the algorithm (30), (31) until the number of reductions is log 2 Nk 1 . • Then we continue
with the reductions of the response and the signal nk in the direction k  1 (again using formulas (30),
(31)). We proceed in this way until all the reductions of the response in all the directions are carried
out, i.e., until the k-dimensional discrete signal of the response is reduced to the only non-zero
element. It represents the value of the determinant of the system matrix hk hence
D  r hk  0,0,K ,0 ,
where
k
r   log 2 Ni
i 1
The final solution is then given by relation
n
xk  k
D
(32)
3. Special linear systems
•Toeplitz, Hilbert, Vandermonde
3.1 Error-free algorithm to solve integer Toeplitz system [8]
•symmetrical Toeplitz arises when solving overdetermined convolution system
•we present an error-free form of Levison algorithm to solve integer
nonsymmetrical Toeplitz system.
•it removes roundoff errors. Their accumulation can, when using classic algorithms, deteriorate or
destroy the solution.
•it retains the speed of original Levison [9] algorithms (proportional to N 2 ).
a1 , a2 ,K , a N 1   x1   y1 
 a0 ,
 a,
 x  y 
a
,
a
,
K
,
a
1
0

1

N

2

 2    2 
 M
M   M  M

    
a
,
a
,
a
,
K
,
a
N 2
N 3
0   xN 
 N 1
 yN 
(33)
or
N
a
j 1
i j
x j  yi , i  1, 2,K ,N
(34)
For integer regular Toeplitz system one can write
N
a
j 1
i j
xj
DN
 yi , i  1, 2,K ,N
(35)
Levison derived the algorithm for fast solution of (33) using recursive procedure solving in each
iteration step the system
M
a
j 1
i j
v(j M )  yi , i  1, 2,K ,M ; M  1, 2,K ,N
(36)
In [8] we derived the algorithm for fast solution of integer Toeplitz system (35) using the following
procedure
M
DM 1  DM  a0   a j b(j M ) ,
j 1
M
( M 1 )
vM

1
yM 1 DM   aM 1 j v(j M )
j 1
M
(M )
a0 DM   aM 1 j bM
1 j
,
j 1
DM 1
( M 1 )
M 1
c
M
 aM 1 DM   aM 1 j c(j M )
j 1
and
( M 1 )
M 1
b
M
 a M 1 DM   a j  M 1b(j M )
j 1
to determine quantities with the index M+1,
(37)
v(j M 1 ) 
1
v(j M ) DM 1  vM( M11 )bM( M1) j  ,
DM
c(j M 1 ) 
1
c(j M ) DM 1  cM( M11 )bM( M1) j 
DM
(38)
and
b(j M 1 ) 
1
b(j M ) DM 1  bM( M11 )cM( M1) j  ,
DM
j  1, 2,K ,M ,
to determine remaining components of the vectors v, c, b, and initial values
D1  a0 , v1( 1 )  y1 , c1( 1 )  a1 , b1( 1 )  a1 .
(39)
From the above given formulas, one can see that all quantities are integers. After some iteration
steps, however, their magnitudes increase rapidly, which complicates the realization of the
calculation. To overcome this problem, we can proceed in several ways:
•to perform the calculation with these long integers what means to watch overflows; the calculation
becomes more time consuming and inconvenient;
•to perform the calculation in floating point form, which means to give up the accuracy of the
algorithm; for ill-conditioned systems, this can give rise to the well-known problems with stability of
the algorithm; and
•to perform the calculation in modulo arithmetic; one does not need to watch overflows, and also
the roundoff errors are removed.
When using modulo arithmetic, we carry out the calculation in different prime modulo classes mi , i
= 1,2, . . . , r, so that it holds [10]
P  m1m2 K mr
where P must satisfy the condition
P  2  max  N N / 2  M ( A )N ,N( N  1 )( N 1 ) / 2  M ( A )N 1  M ( y ) ,
(40)
where
M ( A )  max ai , i   N  1,N  1 ,
M ( y )  max y j ,
j  1,N
The calculation in the given modulo class is independent of other modulo classes and thus this
model of implementation is well suited for parallel computing. By inverse conversion from residual
representation, employing Chinesse theorem [2], [9], one can calculate resulting vector x and
determinant D.
Illustrative example
Find the exact solution of the Toeplitz system
1, 1, 2   x1   1
 3, 1, 1   x    3 

  2  
 2, 3, 1   x3   1 
We initialize the vectors and variables
a p   a1 ,a2 , 0   3, 2 , 0
T
T
a n   a1 ,a2 , 0   1, 2, 0
T
T
y   1,3,1
T
b   1,  ,  
T
c  3,  ,  
T
v   1,  ,   , D  1.
T
With respect to (40) we choose the moduli
m1  5, m2  7, m3  11
Carrying out the first iteration step yields
b(mod 5 )  3, 1,  ; b(mod 7 )  0, 1,  ; b(mod 11 )   4, 1,   ;
T
T
T
c(mod 5 )   0, 3,  ; c(mod 7 )  5, 0,  ; c(mod 11 )  5, 4,   ;
T
T
T
v(mod 5 )   2, 1,  ; v(mod 7 )   2, 6,   ; v(mod 11 )   2, 6,   ;
T
D(mod 5 )  4;
T
D(mod 7 )  4;
T
D(mod 11 )  4.
After the second iteration step (M=N-1=2) the calculation in this example is finished. The resulting
values of vectors and determinant are then as follows
b(mod 5 )  3, 1, 0 ; b(mod 7 )  0, 1, 1 ; b(mod 11 )   4, 1, 4 ;
T
T
T
c(mod 5 )   0, 3, 4 ; c(mod 7 )  5, 0, 4 ; c(mod 11 )  5, 4, 0  ;
T
T
T
v(mod 5 )  1, 3, 2 ; v(mod 7 )   2, 3, 3 ; v(mod 11 )  5, 3, 4  ;
T
D(mod 5 )  3;
T
D(mod 7 )  2;
T
D(mod 11 )  1.
By conversion of the vector v and the determinant D from residue class code one obtains
v  16, 3, 367  , D  23.
T
Positive numbers are represented in the range <0,(P-1)/2> and negative ones in the range
<(P-1)/2+1,P-1>. Then the negative values are, for the values being greater than (P-1)/2, calculated
by subtracting P. In our example the element v(3)=367-385=-18 (367>5.7.11/2). Then the final
solution is
1
T
x  16, 3, 18 .
23
3.2 An algorithm to solve Hilbert system of linear equations exactly [11]
•typical example of extremely ill-conditioned matrices are Hilbert matrices. In literature one can find
the examples of Hilbert matrices with the elements
ai , j 
1
; i, j  1,N
i  j 1
•we shall consider more general Hilbert systems with elements
ai , j 
1
di  j 1
; i, j  1,N
then for N=3 we can write
1
1
1
,
,


b
b
b
2
3
 1
 x1   y1 
1
1
1    
,
,

  x2    y2 
 b2 b3 b4   x   y 
 3  3
1
1
1
,
 ,

b
b
b
4
5
 3
or after exchanging the elements of the vector x
1
1
1
1
1
1 
,
,
 ,

 ,

b
b
b
s
s
s
2
1 
1
2 
 y1   3
 x3   0
 z1   R0 , R1 , R2   z1 
 y    1 , 1 , 1  x    1 , 1 , 1  z    R , R , R  z 
 2 
 2  1
0
1   2 
 2  b
b
b
s
s
s
3
2
0
1 
 y3   4
 x1   1
 z3   R2 , R1 , R0   z3 
1
1
1
1
1
1 
,
,
 ,

 ,

b
b
b
s
s
s
4
3
1
0 
 5
 2
This represents Toeplitz system of linear equations. Assuming that all quantities are integers the
determinant of the matrix can be written in the form of rational fraction. For N=3 one obtains
C
C
1
1
1
1
2
D
D
D 3  2  2 

 1 2 3 2 1  D
s0 s2 s1 s1 s2 s2 s0 s2 s1s0 s1 s2 s1s0 s1 s2
D
where left upper index C denotes numerator and D denominator. Without being interested in the
numerator for denominator of the determinant in general case one can write
D
D  s1 N 1s2 N  2 K sN11s0N s1N 1 L sN2  2 s1N 1
Analogously to integer Toeplitz system we have converted Hilbert system to Toeplitz one composed
of rational fractions and subsequently we have derived the error-free algorithm to solve it. In this
case we introduce pairs of expressions of all quantities for both numerator and denominator,
respectively
First let us determine quantities with the index M+1,
D ( M 1 )
M 1
b
D ( M 1 )
M 1
c

M 1

M 1 i 1
i  M 1

M 1

si
M 1 i 1
i  M 1
si
;
C ( M 1 )
M 1
;
C ( M 1 )
M 1
b
c
C (M )
C
M


bj
DM
D ( M 1 )
 bM 1 

;
D
D (M ) 
 s M 1 DM j 1 s j  M 1 b j 
C (M )
C
M


cj
DM
D ( M 1 )
 cM 1 

;
D
D (M ) 
 sM 1 DM j 1 sM 1 j c j 
M 1
D ( M 1 )
M 1
v
 D cM( M11 )

i 1
M 1
D
DM 1 
 si
i  M
yi
s
i 1
M
D
M 1 i
;
i
;
C ( M 1 )
M 1
v
C (M )
C
C
M


vj
y
D
D ( M 1 )
M 1
M
 vM 1  D

;
D
D (M ) 
 yM 1 DM j 1 sM 1 j v j 
C (M )
C
M


bj
D
C
D
M
DM 1  DM 1  D

;
D (M ) 
 s0 DM j 1 sM 1 j b j 
Then we determine remaining components of the vectors v, c, b, and initial values
j 1
D ( M 1 )
j
b
 D DM 1
s
i 0
 M 1 i
;
j 1
s
C ( M 1 )
j
b

b
i
i 0
DM  C DM 1 Cb(j M ) C bM( M11 ) C cM( M1) j 
 D ( M 1 ) D ( M )  ;
D
D (M )
D
bM 1 cM 1 j 
 DM 1 b j
D ( M 1 ) D
j
C
M
j 1
D ( M 1 )
j
c
 D DM 1
s
i 0
j 1
s
i 0
D ( M 1 )
j
v
M 1i
 c
C ( M 1 )
j
c

i
D ( M 1 )
D ( M 1 )
M 1
j
D ( M 1 )
M 1
v
c
;
;
C ( M 1 )
j
b

DM  C DM 1 C c(j M ) C cM( M11 ) C bM( M1) j 
 D ( M 1 ) D ( M )  ;
D
D (M )
D
cM 1 bM 1 j 
 DM 1 c j
D ( M 1 ) D
j
C
M
c
DM  C DM 1 C v(j M ) C vM( M11 ) C bM( M1) j 
 D ( M 1 ) D ( M )  ;
D
D (M )
D
vM 1 bM 1 j 
 DM 1 v j
D ( M 1 ) D
j
C
M
v
j  1, 2,L ,M ,
where
D
D1  s0 ; C D  1; D v1( 1 )  D y1 ; C v1( 1 )  C y1 ; Db1( 1 )  s1 ; Cb1( 1 )  1; D c1( 1 )  s1 ; C c1( 1 )  1;
3.3 Error-free algorithm to solve Vandermonde system of linear equations [12]
Let us have Vandermonde system
1
1
 y1   1
y   a
a2
a3
 2  1
 y3    a12
a22
a32
  
M
M
 M  M
 yN   a1N 1 a2N 1 a3N 1
L
L
L
L
1   x1 
aN   x2 
aN2   x3   A  x
 
M   M
aNN 1   xN 
(41)
Vandermonde system arises when solving polynomial interpolation problem
It is well known that the determinant D of the matrix A is
N 1
N
D    ( ai  a j )
(42)
j 1 i  j 1
and the elements of inverse matrix can be expressed using polynomial coefficients [12]
N
x  ai
Pj ( x )  
  B j ,k x k 1
k 1
i 1,i  j a j  ai
N
(43)
Analogously to the algorithm given in [13] we determine coefficients of the polynomial
N
P( x )   ( x  ai )(mod M )  x N  cN x N 1  L  c2 x  c1(mod M )
i 1
(44)
Polynomials from (43) modulo M can be expressed
N
Pj ( x ) 
P( x )

( x  aj )
1
(mod M ) 
N
 (a
i 1,i  j
j
 ai )
z
k 1
N
j ,k
 (a
i 1,i  j
x k 1
(mod M )
j
 ai )
(45)
Then one can derive
z j ,N  1,
(46)
z j ,N 1  ( cN  z j ,N  a j )(mod M ),
M
z j ,N i  ( cN i 1  z j ,N i 1  a j )(mod M )
i  1,N  1 ,
j  1,N .
Resulting inverse matrix modulo M is
 d1
0
1
A (mod M )  
M

0
0
L
d2 L
M
0 L
0
0 
M

dN 
1
 z1,1 z1,2 L
z
 2 ,1 z2 ,2 L
 M M

 z N ,1 z N ,2 L
z1,N 
z2 ,N 
(mod M )  BM
M

z N ,N 
(47)
where
dj 
N
 (a
i 1,i  j
j
 ai )(mod M )
We can employ iterative scheme of error-free solution of linear equation system from section 2.3.
Illustrative example
1,1, 1, 1   x1 
0 
1,3, 5, 4   x 
 

  2   1 
1,9 , 25, 16   x3 
2

 
 
1, 27 ,125, 64   x4 
0 
The determinant of the matrix, according to (42), is -48. We shall use modulus M = 5. Then
DM = 2. Using the relations (45), (46), (47) yields inverse matrix in modulo class 5
24,0 ,0, 060, 47,12, 1
0, 2, 3, 1
BM  0, 4 ,0, 0 20, 29,10, 1(mod 5)  0, 1, 0, 4 
0, 0 ,8, 0 12, 19,8, 1
1, 3, 4, 2 
0, 0 ,0,  3 15, 23,9, 1
0, 4, 3, 3
Then the procedure to calculate exact solution is as follows.
a. ym0 = y=[0,1,2,0]T .
b.
0, 2, 3, 10
3
x M = BM ym0 (mod 5)  0, 1, 0, 4 1 (mod 5)  1 
1, 3, 4, 2 2
1 
0, 4 ,3, 30
0
c. x0 =DMxM (mod 5)=2.[3,1,1,0]T =[1,2,2,0]T.
d. x = [1,2,2,0] T.
e. We calculate vectors
'
0
1 1 
1, 1, 1,
5

1
,
3
,
5
,
4
2





 17 
25, 162
1, 9,
69 


1, 27, 125, 64

0

305

y
= Ax 0
y1
0

5

1 
1 48
17



=

 13
69 
33
5 96




0

305

61



ym1 = [4,2,2,4]T.
f. Let j = 1.
g.
x1
0, 2, 3, 14
4
0
,
1
,
0
,
4
2




= B M ym1(mod 5) 
(mod 5)  3
1, 3, 4, 2 2
1 



0, 4 ,3, 3

4

1 
Suppose that
positive
numbers
are represented
in
the
range <0,(M-1)/2> and negative
numbers in the range <(M-1)/2+1,M-1>. Then negative values are, for numbers greater
than (M-1)/2, obtained by subtracting the modulus
x1 = [-1, -2, 1, 1]T.
h.
1 
1
4
x = 2  5  2  8
2
1 
7 



0

1 

5 

i. Calculate vectors
y1’ = Ax1 = [-1, 2, 22, 134]T
y2 = 5-1 [0, 3, -11, -39] T
ym2 = [0, 2, 4, 1]T.
Now we continue in the next iteration (j = 2) in the step g.
g. x2 =BMym2 (mod 5) = [2,1,4,3]T .
After adjustement of elements greater than (M-1)/2 we have
x2 = [2,1,-1,-2]T .
h.
4
2 
46 
x = 8  52  1   17 
7 
1
18



5 

2

45

i. Calculate vectors
y2’ = Ax2 = [0,-8,-46,-224]T ,
y3 = [0,1,7,37]
T
,
ym3 = [0,1,2,2]T.
We continue in the next iteration (j = 3) in the step g.
g. x3 =[0,-1,0,1]T .
h.
46 
0 
46

17

1

108






3
x =
5 

18
0 
18 




45

1 

80

i. Calculate vectors
y3’ = [0,1,7,37]T,
y4 = [0,0,0,0]T.
The zero vector y4 indicates the end of iterations. The result is
46

1 108
x =
.
-48 18 


80

4. General linear systems
4.1 Parallel error-free algorithm [14, 15]
Let us have system of linear equations, where all the elements of the matrix A and vector y are
integers
N
y( n )   a( n,m )x( m ), n  1, 2,L , N
or
m 1
 a1,1 , a1,2 , a1,3 ,K , a1,N   x1   y1 
 a , a , a ,K , a   x   y 
2 ,2
2 ,3
2 ,N   2 
 2 ,1

  2   Ax=y
 M
M   M  M

    
a
,
a
,
a
,
K
,
a
N ,2
N ,3
N ,N   xN 
 yN 
 N ,1
Let us split the calculation into several residue classes satisfying.
P  m1m2 K mr
where P must satisfy the condition
P  2  max  N N / 2  M ( A )N ,N( N  1 )( N 1 ) / 2  M ( A )N 1  M ( y ) ,
where
M ( A )  max ai , j , i, j  1,N ,
M ( y )  max yi , i  1,N
Then in each residue class we can carry out separately the calculations.
Ax(mod mi )=y , i  1,r
(48)
Calculations in residue classes can be carried out in parallel
Gauss-Jordan elimination in residue class m can be described by the following procedure
Let
0
ai, j   ai , j  (mod m); i  1, 2, K , n , j  1, 2, K , n  1 .
For k  1, 2,K , n (k denotes elimination step) do:
1. search for such i  c f  f  k  that ai ,1  0, ck  i,
2.
3.


 k 1
 k 1 1 

ai , j  ai , j 1  ai ,1
(mod m),


for l  1, 2,K , n and l  i do:
k 
j  1, 2,K , n  1  k.
k
k 1
k 1
k
al, j  al, j 1  al,1   ai,1  (mod m),
j  1, 2,K , n  1  k.
(49)
(50)
The determinant in the residue class m is obtained as the modulo product of the elements ai,1
(49))
k 1
n

J 
 k 1  
D
(mod
m
)

D


1
a
 
   k ,1   (mod m),
m
 k 1


(see
(51)
where J is the total number of exchanges of the elements ck which create an increasing sequence. Then,
the sought solution of (48) in the residue class m is given by
 x (mod m)  xm  
D 
1
xm  (mod m)   Dm  xm  (mod m).
D
D 
(52)
It can be noticed that by executing each elimination step the columns of the matrix are shifted to the left (the
contents of the rightmost column being irrelevant), so that after finishing the elimination the vector
 x (mod m) is placed in the first column of the memory matrix (elements ai,1 , i 1, 2,K , n ).
The described algorithm can be easily implemented in the residual processor, presented in block
diagram in Fig. 2.
The residual processor contains the memory matrix M, the arithmetic units AUD , AU2 to AUn+2, and
the unit denoted as ROMs. A jth column of the memory matrix is connected by its serial output to AUj•
The unit ROMs comprises two ROM memories, one with negative numbers and one with inverse
numbers in the residue class m. From the first column of the matrix M the values ai,1
k 1
(see (49)) and
al,1k 1 (see (50)) are fed through the e bit internal data bus IDB to the unit ROMs and are used as
addresses to find the appropriate inverse and negative numbers. The inverse and negative numbers are
supposed to be written into each residual processor’s ROMs beforehand in correspondence to the
ascending sequence of the addresses.
The arithmetic units AU2 to AUn+2
serve to perform the calculations (49), (50). i.e. they are
designed to carry out the operations of the residue class addition and multiplication.
Finally. the arithmetic unit AUD performs the calculation of the determinant D modulo m according
to (51).
The shift of the columns to the left in the elimination step is achieved by interconnec tions of the matrix
columns and the arithmetic units by serial input and output buses SI, SO.
Conversion from residue class code can be done according to the following algorithm:
After the calculation of the solution of a linear equations set in the modular arithmetic is terminated the
n  1 element vectors
 D  (mod m ),  x  (mod m
k
are
prepared
in
the
1
arithmetical
k
), K ,  xn  (mod mk )  ,
units
k  1, 2, K , r
AU 2 , AU 3 , K , AUn+2
of
the
(53)
residual
processors
RP1 , RP2 ,K , RPr . The resulting integer values D, x1 , x2 ,K , xn can be obtained by the inverse conversion
of the vectors (53) from the residue class code. We shall assume that the resulting values are formed in the
floating point processor FFP in the control unit CU as the n  1 -element vector of the floating point
numbers.
Since we use the known algorithm (given e.g. in [16]) we describe it very briefly from mathematical
point of view. We suppose we have a vector of moduli
0   m1 , m2 ,K , mr 
(54)
and a vector of the residue representation of the integer x j


t j ,1   x j  mod m1 ,  x j  mod m2 ,K ,  x j  mod mr .
(55)
The elements of the vector (54) are written in the ROM memory and the elements of vector (55) in the
arithmetical units AU j 1 of the residual processors RPk , k  1, 2,K , r . Further. let us denote
k   mk 1 , mk  2 ,K , mr 
t j , k   t j , k  k  , t j , k  k  1 , K , t j , k  r   ,


d j , k  t j , k  k  mod mk ,K , t j , k  k  mod mr .
If we define the vector t j , k 1 to be
1

t j , k 1  
 t j , k  d j , k   mk  mod  k ,
then the resulting sought number x j is given by


x j  t j ,1 1  m1 t j ,2  2   m2 
t j ,3  3  L  mr 1  t j , r  r  
 .
Repeating the computation using this algorithm for
resulting vector
j  1, 2,K , n  1
(56)
in the control unit CU we obtain the
x1 , x2 ,K , xn and determinant D . Let us illustrate the given algorithm by an example with
0   3, 5, 7  , and
t j ,1  1, 2,5 - see Table 1.
Table 1
0
m1  3
m2  5
m3  7
t j ,1
1
2
5
 d j ,1
1
1
1
1
t
j ,1
 d j ,1 
0
1
1
2
5
2
6
d j ,2
2
2
0
4
j ,2
 d j ,2

1
2
3
*m
t j ,3
Then, using (56), the number
5
x j is
x j  t j ,1 1  m1 
t j ,2  2   m2  t j ,3  3 
  1  3   2  5  5  82.
subtraction
4
*m
t j ,2
t
operation
multiplication
subtraction
multiplication
4.2 Sequential algorithm to solve general system of linear equations [17]
So far we have mentioned two basic algorithm classes to solve linear equations exactly
• iterative algorithms (applied in section 2.1 – Fermat transform, section 3.3 – Vandermonde
system)
• parallel algorithms (applied in section 3.1 – Toeplitz system, section 4.1 – general system of
linear equations)
In this section we want to demonstrate that sequential iterative algorithm can be applied also
for general systems of linear integer equations. Let us define the following algorithm:
Algorithm A
a. Let y m0  y.
b. Calculate vector x M
xM  A1ym0  mod M  .
c. It holds
x0  DM xM  mod M  .
d. Let x  x0 .
e. Calculate vectors
y '0  Ax0 ,
y1 
1
y m0  D  y '0  ,

M
(57)
ym1  y1  mod M  .
If y 1 equals zero vector the calculation is finished, if not continue in f.
f. j  1.
g. Calculate
x j  A1ym1  mod M  .
h. For i  0 ,1,K ,N  1 (N is the size of vectors x, y ) calculate
(58)
x i   x i   x j i  M j .
i. Calculate vectors
y'j  Ax j ,
y j+1 
y j  y'j
M
,
ym,j+1  y j+1  mod M  .
j. If for all i  0,1,K ,N 1, y j 1 i   0, finish the calculation. If not, increment j and repeat
the algorithm from point g. on.
The resulting solution of system (1) is simply calculated as
x
x .
D
Then determinant of the system of linear equations can be calculated using the formula
j 1


D    a  j, j    a  j,i   k j  i   ,
i 0
j 1 

N 1
(59)
where the coefficient vectors k j are calculated by exact solution of sets
Aj , j  k j  Aj ,1
j  1, 2,K ,N  1,
(60)
using the algorithm A
Next problem in the algorithm A is calculation of the inverse matrix in modulo arithmetic. Let us
define iterative procedure where:
Aj 1, j 1
 Aj , j

 A1, j
Aj ,1 
,
A1,1 
Bj, j
A j1, j1  mod M   
 B1, j
1
B j ,1 
,

B1,1 
(61)
Then one can derive
1
 mod M  ,
Bj,1   Aj,1j  Aj,1  B11,  mod M  ,
B1, j  B11,  A1, j  Aj,1j  mod M  ,
Bj, j  Aj,1j  Bj,1  B11,1  B1, j  mod M  .
B11,   A11,  A1, j  Aj ,1j  Aj ,1 
(63)
4.3 Sequential error-free algorithm to solve system of polynomial equations [18]
Sequential iterative algorithm can be successively applied also for polynomial systems of linear
integer equations:
A  s   x  s   y  s  , ai, j  s  
ai, j  s p 1  L  1ai, j  s1  0ai, j , yi  s  
p 1
p 1
yi  s p 1  L  1 yi  s1  0 yi , (64)
Assuming the coefficients being integers, the solution of the system given by (64) can be expressed
as
xs 
1
k
  s p  1  x0  s    s p  1  x1  s   L   s p  1  x k  s   ,

D  s  
1
(65)
and vectors xi ,i  0,1,K ,k are:
xi  s    M 0  xi 0  s   M 1  xi1  s   L  M m  xim  s   ,
(66)
where M is prime modulus, D is the determinant of matrix A and m, k are finite integers. Let us
suppose for the moment that the determinant D  s  (polynomial) is known. We denote polynomial
modulus as
P  s   s p 1 .
The algorithms for the determinant and inverse matrix of polynomial system can be derived
analogously to relations (59-63). For details we refer to [18]. New problem is the calculation of
inverse polynomial. In what follows we outline this algorithm.
We have to find polynomial b  s  to polynomial a  s  so that
a  s   b  s    mod P  s   (mod M )  1
(67)
According to [2], the cyclic convolution using polynomial algebra concept is
p 1
cl   ai  bl i ,
(68)
i 0
where indices l  i
are calculated modulo p
From what is given above, it follows that solution of equations (67) represents the solution of linear
system
 a 0 ,
a  p  1 , L a 1  b  0  
1 


 
0 
a
1
,
a
0
,
L
a
2
b
1












(mod M )    ,
(69)





M
M
M
M
M


 
 
 a  p  1 , a  p  2  , L a  0   b  p  1 
0 
in residual class M.
Inverse polynomial b  s  can be obtained by successive reductions of polynomial a  s  or vector
a . For the r+1 -st reduction it holds
r 1
a  2r 1  l  
 N / 2  1
r

r
i 0
a  2r  i   r a  m    1 (mod M ),
i
where
 N 
m   2r 1  l  2r  i  (mod N ); l  0, r 1   1 ,
2 
and number of reduction r  0,1,K ,log2 N . Let
0
n  1, 0,L ,0 ,
T
 N / 2  1
r
r 1
n l  

i 0
where
r
and
n  2r  i   r a  m    1 ( mod M ),
i
m   l  2r  i  (mod N ); l  0,n  1 ,
and number of reduction r  0,1,K ,log 2 N  1. Resulting solution of polynomial inversion is
b
where t  log 2 N.
t
n
 (mod M ),
t
a  0
Illustrative example
Let us have linear system consisting of polynomials:
 s 2  1
 s  1, 3s 
.
 2s, s 2  1  x  s   
2




We use polynomial modulus
P  s   s3  1
and numerical modulus M  5 .
We assume that we know determinant
D  s   s3  5s 2  s  1,
determinant in residue class P  s  and residue class M


Dpm  s   D  s  mod  s3  1  mod 5  s  2 ,
(70)
and inverse matrix in residue class P  s  and residue class M
 s  3,
4 s 2  2 s  3
B  s   A  s   mod  s  1  mod 5    2
.
2
s

3
s

2
,
s

3
s


1

3

We calculate x p  s 
 s  3,
s2 
4s 2  2s  3  s 2  1
3
xp s   2

  mod  s  1   mod 5    .
2
s

3
s

2
,
s

3
s
2

 

0 


Using
Dpm  s  from (70), we get:
s2 
x00  s   Dpm  s     mod
0 

s
3
 1

  mod 5  20s
2


.

We initialize vectors and control variables
j  0,
 s 5  5s 4  2s 3  4s 2  s  1
 s 2  1
3
2
Ts  y s  D s  

s

5
s

s

1

  2s3  10s 2  2s  2
,
 


 2 


k  0,
 2s 2  1
xs  
.
 0

We calculate vectors
3s   2s 2  1  2s 3  2s 2  s  1

,

3
s 2  1
4
s

2
s

  0
 


 s  1,
y '00  s   A  s   x00  s   
 2s,
 s 5  5s 4  6s 2 
T  s   T  s   y '00  s   M 0  
,
3
2

2
s

10
s

2




y 01  s  

T  s  mod
s
3
 1
5
  1   5s

5 

 5s    s 2  s 
,

2 
 10 s 2 
 

  2s 
2
 4s 2  4s 
y p,01  s   y 01  s  mod 5   
.
2
 3s



The vector y 01  s   0, thus we set k = 1

x01  s   B  s   y p,01  s  mod  s 3  1

s  3,
 2
 s  3s  2,
  mod 5 
2
4s 2  2s  3  4s  4s 
3
 mod  s  1

2
2
s  3s  

 3s

 4s 
 5
    mod 5   
.
 0
 0

  mod 5 
If any coefficient of the solution is greater than  M  1 / 2 , it is considered negative and in
accordance with rules of modular arithmetic its negative value is calculated by subtracting modulus
M.
We continue with
 2s 2  1   s  1  2s 2  5s  1
xs  
    5  
.
0
0
0

  


NOTE.
Further, we calculate appropriate vectors
2
 s  1, 3s   s  s  s 
y  s   a  s   x01  s   
,
 0   
2
2 
2
s,
s

1

     2s 
'
01
 s5  5s 4  s 2  5s 
Ts  Ts  5  y s  
,
3

2
s

2


1
y 02  s  

'
01
T  s  mod  s 3  1
yp,02  s   0.
5
  0,
The vector yp,02  s   0. The vector T  s   0 , therefore, we set control variables j  1, k  0 ..
Then we calculate
T  s   s 2  5s 
s2 
Ts  3

y p,10  s   T  s  mod 5    .
;
s 1   2 
3 
Further we calculate next particular solution
 s3
4s 2  2s  3  s 2 
s
s
3
x10  s    2


mod
s

1

mod
5


mod
5







  
3
 2 ,
2
s

3
s

2
s

3
s
3
 
 

  


which we add to final solution
 s 4  2s 2  6s  1
 2s 2  5s  1  s  3
xs  
.
      s  1  
3
0
  2s  2


 2
Again we repeat and calculate
 s  1, 3s   s   s 2  5s 
y  s   A  s   x10  s   
,
   2  
2
2
s,
s

1

2

   

'
10
'
T  s   T  s   y10
 s   50  0,
y11  s   0,
yp,11  s   0,
y11  s  as well as vector
calculation. The resulting solution is
The vector
T  s  equal zero vectors what means that we finish the
4
2

s

2
s
 6s  1
1
1
xs 
 xs  3

.
2
3
D s
s  5s  s  1   2s  2

5. Volterra systems
Direct generalization of convolution systems for nonlinear systems
N 1
N 1 N 1
N 1 N 1 N 1
l1  0
l1  0 l2  0
l1  0 l2  0 l3  0
y ( j )   h1 (l1 ) x( j  l1 )   h2 (l1 , l2 ) x( j  l1 ) x( j  l2 )   h3 (l1 , l2 , l3 ) x( j  l1 ) x( j  l2 ) x ( j  l3 )
 y1 ( j )  y2 ( j )  y3 ( j ) 
,
j  0,1,
(71)
, N 1
Several problems:
1. Determination of Volterra kernels [19]
2. Calculation of the output of the nonlinear Volterra filter [20]
3. Determination of inverse kernels to the given nonlinear Volterra system [21]
Let us again employ polynomial algebra
N 1
N 1 N 1
X ( z )   x( l )z ; H i ( z1 ,z2 ,L ,zi )  L
l
l 0
l1  0 l2  0
N 1
 h ( z ,z ,L ,z )z
i
1
2
i
z L zili
l1 l2
1 2
li
Then for the Volterra kernel of the i-th degree we can write
p
Yi ( z1 z2 L zi )
Y ( z ,z ,L ,zi )
H i ( z1 ,z2 ,L ,zi ) 
 i 1 2
X ( z1 )X ( z2 )L X ( zi )
D
(72)
Illustrative example
Let N=2, i=2 and
y 2  3, 4 ; x= 1, 2
T
T
Then according to (72)
3  4 z1 z2
( 3  4 z1 z2 )( 1  2 z1 )( 1  2 z2 )
H 2 ( z1 ,z2 ) 


( 1  2 z1 )( 1  2 z2 ) ( 1  2 z1 )( 1  2 z1 )( 1  2 z2 )( 1  2 z2 )
19  14 z1  14 z2  16 z1 z2 19  14 z1  14 z2  16 z1 z2

2
2 2
(1  2 )
9
The resulting kernel of the second degree is
1  19 14
h2  
9  14 16 
The product X(z)X(-z) is called the reduction of the polynomial X(z) with respect to the variable z.
Conclusion
In the contribution we have presented several error-free algorithms to solve one- and
multidimensional convolution systems based on Fermat number theoretical transform. The main
significance of the use of modulo arithmetic is eliminating rounding-off and truncation errors.
In the contribution we have derived the error-free algorithm to solve convolution systems that is
based on polynomial algebra concept. This form of k-dimensional deconvolution algorithm permits
to express it as a sequence of k-dimensional “autoconvolutions” of the response signal and “crossconvolutions” of the response and the output signal.
Further in the contribution we have extended the error-free algorithms to special linear systems,
e.g. Toeplitz, Hilbert, Vandermonde systems. We continued with general systems of linear
equations. In principle the algorithms proposed in the contribution can be divided into two groups
• iterative (using one modulus – one residue class)
• parallel (using several moduli – several residue classes and Chinese theorem
We have extended the iterative algorithm also for the linear system of polynomial equations. At
the end of the contribution we outlined the possible extension of the application of error-free
algorithms for nonlinear Volterra systems.
References:
[1] N. Ahmed and K.K. Rao: Orthogonal Transforms for Digital Signal Processing. Springer-Verlag,
1975.
[2] H. J. Nussbaumer: Fast Fourier Transform and Convolution Algorithms. Springer-Verlag, 1981.
[3] M. Morháč: Precise Deconvolution Using the Fermat Number Transform. Computer and
Mathematics with Applications. Vol. 12A, No. 3, pp. 319-329, 1986.
[4] Morháč M.: Error-free deconvolution based on cyclic determinant calculation approach.
Computer Mathematics, Vol. 49, pp.41-51, 1993,
[5] M. Morháč: k-dimensional Error-free Deconvolution Using the Fermat Number Transform.
Computers and Mathematics with Applications. Vol.18, No.12, pp.1023-1032, 1989.
[6] M. Morháč: Precise Multidimensional Deconvolution Using the Polynomial Algebra Concept.
International J. Computer Mathematics, Vol.32, pp.13-26, 1990.
[7] M. Morháč, V. Matoušek: Exact algorithm of multidimensional circulant deconvolution. Applied
Mathematics and Computation, Vol. 164/1 (2005), 155-166.
[8] M. Morháč: An error-free Levison algorithm to solve integer Toeplitz system. Applied
Mathematics and Computation, Vol. 61, No.2-3, pp. 135-149, 1994.
[9] R. E. Blahut: Fast Algorithms for Digital Signal Processing, IBM Corporation, Owego, NY, 1985.
[10] M. Newman: Solving equations exactly, Nut. Bureau Standards 71B:171-179 (1967).
[11] M. Morháč: An algorithm to solve Hilbert systems of linear equations precisely. Applied
Mathematics and Computation, Vol. 73, No.2-3, Dec. 95, pp.209-229.
[12] M. Morháč: An iterative error-free algorithm to solve Vandermonde systems. Applied
Mathematics and Computation. Vol. 117/1, 2001, pp.45-54.
[13] W. H. Press et al.: Numerical Recipes, Cambridge University Press, Cambridge, 1986.
[14] M. Morháč, R. Lórencz: A modular system for solving linear equations exactly. I. Architecture
and numerical algorithms.Computers and Artificial Intelligence, Vol.11, No.4, 1992, pp.351-361.
[15] R. Lórencz, M. Morháč: A modular system for solving linear equations exactly. II. Hardware
realization.Computers and Artificial Intelligence, Vol.11, No.5, 1992, pp.497-507
[16] R.T.Gregory, E.V.Krishnamurthy: Methods and applications of error-free computation. SpringerVerlag, New York, Berlin, Heidelberg, Tokyo 1984.
[17] M. Morháč: One-Modulus Residue Arithmetic Algorithm to Solve Linear Equations Exactly.
Mathematical and Computer Modelling, Vol.19, No.12, pp.95-105, 1994.
[18] M. Morháč: An Error-Free Algorithm to Solve Linear System of Polynomial Equations.
Mathematical and Computer Modelling, Vol.19, No.12, pp.85-93, 1994.
[19] M. Morháč : Fast Error-free Algorithm for the Determination of Kernels of the Periodical
Volterra Representation.Applied Mathematics and Computation, Vol.38, No.2, 1990, pp. 87-101.
[20] M. Morháč: A Fast Algorithm of Nonlinear Volterra Filtering.IEEE Transactions on Signal
Processing, Vol.39, No.10, 1991, pp. 2353-2356.
[21] M. Morháč: Determination of Inverse Volterra Kernels in Nonlinear Discrete Systems. Nonlinear
Analysis, Theory, Methods & Applications. Vol.15, No.3, pp.269-281, 1990.