Download Kronecker Products and Shuffle Algebra

116
I E E E T R A N S A C T I O N S O N C O M P U T E R S , V O L . C-30,
N O . 2, F E B R U A R Y
1981
Kronecker Products and Shuffle Algebra
MARC DAVIO
Abstract—The paper relates three classical concepts, viz. mixed
radix number system, Kronecker product of matrices, and perfect
shuffle. It presents an algebra which describes the hardware organi­
zation of the computation of a product Mv, where Μ is a matrix in
Kronecker product form and ν is a vector. The algebraic formalism
describes both the blockwise structure of the computation and the
various possible connection patterns.
Index Terms—Kronecker product of matrices, mixed radix number
systems, perfect shuffle.
C
I.
INTRODUCTION
E L L U L A R logic has, for a long time, attracted the at­
tention of designers. In the recent past, arrays of logical
operators showing up a regular connection pattern, such as
arithmetic arrays, have raised up a renewed interest in the
perspective of their implementation in LSI form.
The present paper addresses a class of circuits whose aim
is to compute the transform w = Mv of a vector £ by a matrix
Μ when the latter has a Kronecker product (or direct product)
structure. The interest of this structure is that it corresponds
to a decomposition of the circuit computing w: in many prac­
tical instances, the resulting circuit will consist of a regular
interconnection of identical operators.
Such a situation is encountered in quite a number of appli­
cation fields viz. H a d a m a r d transform [1], discrete Fourier
transform [ 2 ] - [ 6 ] , connection and substitution networks
[7]-[9], [20], canonical expansion of discrete functions [10],
[11], • · · . It is only possible to give here a small sample of the
subject literature and we tried to select, among the relevant
references, those whose formalism was the closest to the one
we use here. For additional references, refer, for example, to
[12] and [13].
It is interesting to recall here some of the milestones in the
development of the underlying theory:
1) discovery of the constant geometry networks computing
the H a d a m a r d transform (1958, [2]),
2) quantitative estimation of the advantages obtained from
the Kronecker product structure, both in hardware investment
and in storage requirement (1963, [10]),
3) description of the fast Fourier transform ( F F T ) as a
product of transformation and of permutation matrices (1968,
[4]), and
4) arithmetic description of the perfect shuffle establishing
its relation with the binary number system (1971, [8]).
Most often these results have been obtained in a specific
context (say the discrete transform theory) and are little or not
Manuscript received April 2,1979; revised March 10, 1980 and September
10, 1980.
The author is with Philips Research Laboratory, Brussels, Belgium.
known in other possible application contexts (say permutation
networks). The main result of the present paper is to establish
a common background for all these problems; it is achieved
thanks to three basic concepts: the mixed radix number rep­
resentation systems, the Kronecker product of matrices, and
the (generalized) perfect shuffle. While these concepts are
classical, their tight relationships probably never have been
clearly stated. This is the subject matter of Section II, which
furthermore lays the basis of a shuffle algebra and establishes
a set of relations describing commonly encountered connection
patterns. Section III is devoted to the factorization of Kro­
necker products. It first presents a decomposition theorem,
which extends to Kronecker products of arbitrary matrices a
factorization of Kronecker powers of square matrices first
described by Lechner [10]. It next uses the shuffle algebra to
derive from the basic decomposition theorem a number of
equivalent circuits. The interconnection patterns of the Cooley-Tukey [3] and of the Yates [14], Good [2] types appear
as particular cases of the obtained circuits.
II. B A S I C C O N C E P T S
A. Mixed Radix Number Representation
Systems
1) Introductory example: If we wish to convert into seconds
a duration such as 3 days 7 h 22 min 4 s (written [3, 7, 22,4]),
we use the formula
3.86400 + 7.3600 + 22.60 + 4 = 285724.
The obtained number of seconds appear as a weighted sum of
the duration [3, 7, 22, 4 ] . The weights are given by the vector
[86400, 3600, 60, 1].
2) Definition and representation theorem: This familiar
situation is imbedded in the concept of mixed radix number
representation system that we now define formally.
Consider an integer valued vector
b= [b -
,b b ];
n h
u
bi>2.
0
(1)
This vector is called basis vector. Build, from this vector a
second vector
w = [w , w n
• • •, w , w ]
n h
x
0
(2)
called weight vector and defined by
i-\
w
= 1; Wi = Wi-xbi-χ
0
= Π bj\
is
( 1 , 2 , · · · , n}.
(3)
7=0
Theorem
0018-9340/81/0200-0116500.75
1: Any integer A such that
0^A<w
n
© 1981 I E E E
(4)
DAVIO: K R O N E C K E R P R O D U C T S
117
has a unique representation
two following
conditions:
[a -\,
, a\, ao] satisfying
n
the
(5)
i=0
2) O^atKbi;
\fie
{0, 1, · · , η - 1}.
(6)
Theorem 1 is merely restated here for the sake of com­
pleteness and its proof will not be reproduced. It may be found
in [15] or [16]. The vector [a -\, · · ·, a\, ao] is the mixed
radix representation of A with respect to the basis \b -\,
,
bu bo\. If b
b\ - · = b -\ = b, the corresponding rep­
resentation is called radix b representation.
Outside of the
present context, mixed radix representations are also used in
residue arithmetic. See, for example, [17].
n
n
=
0
n
B. Kronecker Product of
Matrices
1) Introductory remarks: The H a d a m a r d transform h(v)
of the vector ν = [vo, v\, υ2,
is the vector [Ao, h\, h , Λ3]
computed according to the familiar matrix product
2
ho
1
hi
1 -1
h
1
h
1 -1
3
1
1
1
1
1 -1
v\
•1 - 1
•1
1
L y
(7)
3
or, in short,
h = Hv.
(8)
The computation of h from ν according to (7) apparently
requires four scalar products, each involving three additions.
For vectors ν having 2 components, the computation of the
Hadamard transform would require 2 scalar products, each
involving 2 — 1 additions.
The particular block structure of the transformation matrix,
suggested by the dotted lines in (7), allows one to perform the
computation according to a better suited algorithm, using no
more than n.2 ~
additions or subtractions. When pro­
grammed on a digital computer, the algorithm only requires
2 cells of working storage [10].
The matrix structure taken to advantage in order to achieve
the above improvements is that of Kronecker product of ma­
trices. Basically, the Kronecker product Μ Θ Ν of the two
matrices
n
n
n
n
x
n
M =
a
b
;
N =
A
C
Β
=
aB
bA
bB-
aC
aD
bC
bD
cA
cB
dA
dB
•cC
cD
dC
dD.
Γ
Xm
0
D
•aA
1
.1-1
The addition and multiplication involved in the two con­
sidered types of matrix products (Kronecker and usual) are,
in the case of the H a d a m a r d transform, the addition and
multiplication of real numbers. A similar situation is observed
however in other contexts involving other additive and multi­
plicative laws. For example, if one expresses the canonical
disjunctive expansion of a Boolean function in Kronecker
product form, the underlying operations will be the Boolean
sum and product. Similarly, the canonical ring sum expansions
of Boolean functions, sometimes known as the Reed-Muller
expansions [11], will be expressed by means of the modulo two
sum and product. Finally, the computation of the discrete
Fourier transform is expressed with the addition and multi­
plication of complex numbers. To cover these various situa­
tions, we shall present the definition and properties of the
Kronecker product in terms of an abstract algebraic struc­
ture.
2) Definition and properties of the Kronecker
product:
Consider an algebraic system (B, + , ) involving the set Β and
two operations, the addition ( + ) and the multiplication ( or
no symbol) satisfying the following axioms (V·*, y, z, e B).
1) (associativity)
χ + (y + z) = (x + y) + z;
x(yz) = (xy)z.
2)
(commutativity)
χ -I- y = y -μ χ; χγ = y
3)
(distributivity)
x(y + ζ) = xy + xz.
4) (neutral
elements)
3 0 , 1 e B:0 + x = χ; l. χ = χ.
Observe that our algebraic system could be a complete
distributive lattice or a commutative ring with unit. A (r, c)
matrix Μ over Β is a set of rc elements of Β disposed in r rows
and c columns. The rows and columns of Μ are numbered
from 0 to (r — 1) and from 0 to (c — 1), respectively. The
matrix element belonging to row number i and to column
number j is represented by m(i,j). A (b, 6)-matrix is a square
matrix of order b. The axioms governing the algebraic system
Β allow one to define in the usual way the addition of matrices
( + ) and the (usual) product M\ · M or M\M
of an ( r
ci)-matrix M\ by a ( r , c o ) - m a t r i x M . The addition of ma­
trices is both associative and commutative. The product of
matrices is associative and distributive over the addition.
Furthermore, one may define the unit matrix of order b, \
by
0
is given by
M®N
2
4
n-\
2
their storage will require 2p memory locations, while the
storage of their Kronecker product would require p double
precision memory locations. Similarly, when computing the
matrix product h = Hv, we shall only perform smaller matrix
products involving the Kronecker factors of H; these factors
have the form
0
1?
0
b
U(/.y)=l
(9)
and the key point to the understanding of the new algorithm
to compute h will rest upon the relationship between the
Kronecker product and the usual product of matrices. That
property will appear as Theorem 3 in the next section.
Observe that if Μ and Ν are two square ρ Χ ρ matrices,
iff/ = 7
(10)
= 0
iff/ ^ y
and check that for any (r, c)-matrix Μ
\
Μ = Μ \ = Μ.
(11)
All these facts are elementary consequences of our axioms.
The Kronecker product Μ\ <g> M of an (r cj)-matrix M\
by an ( r , C o ) - m a t r i x M is an (r\r , c\c )-matrix
Μ defined
by
r
c
0
0
0
0
u
0
118
IEEE
m(i,j)
=
m\U\J\)mo(i Jo);
0
i = iVo + io'J =j\Co
in)
+ jo­
The row and column indexes of the Kronecker product Μ thus
have the representations [i\Jo] and \j\Jo] in the mixed radix
systems with basis vectors [r\, r ] and [c\, Co], respectively.
The block structure of Μ is fairly apparent: /1 and j\ are the
block coordinates, while /'o and j are the element coordinates
0
0
within the ( / 1 , 7 1 )-block.
Consider as an example the Kronecker product C = A ® Β
of a (2, 3)-matrix A by a (4, 2)-matrix Β
' boo bo\'
A=
0oo 001 002
a\o a\\
;
012,
b\o bu
B =
bio
bix
•*30
*3I
(13)
-1
TRANSACTIONS
ON COMPUTERS,
V O L . C-30,
N O . 2,
FEBRUARY
1981
These properties are now classical and their proofs may be
found in [11], [18]; they have been restated here for reference.
The most important of these properties is probably contained
in (17), which relates the ordinary and the Kronecker matrix
products. That property contains the essence of the simplifying
mechanism mentioned in Section II-B1. Basically, this
mechanism may be described as follows: if, in the computation
of the matrix product w = M.g, the transformation matrix Μ
may be expressed as the Kronecker product M i ® Mo, then,
it is possible to compute w from independent transformations
described by the matrices M i and M ; (17), in fact, contains
a decomposition process: the latter will be illustrated in Section
III. Equation (17) may be extended in various ways that will
be described in Theorem 3 below. Before stating this theorem,
we need some additional notations.
The Kronecker product
0
C is thus the (8, 6)-matrix
COO C0\ C02 C03 C 4 C05
C\QC\\ C 1 2 C 1 3 C 1 4 C 1 5
000*00
000*01
001*00
001*01
002*00
000*10
000*11
001*10
001*11
002*10
002*11
C oC \ ^22 C C 4 C
25
000*20
000*21
001*20
001*21
002*20
002*21
^30^31
C32 C33 C34 C35
000*30
000*31
001*30
001*31
002*30
002*31
012*00
012*01
0
2
2
23
2
002*01
C =
C40 C41 C42 C43 C44 C45
0 10*00 0 1 *01
cso c \ csi C53 C54 C55
C60 Ce 1 C 2 <?63 C(>4 ^ 6 5
010*10
S
010*20 010*21
[_0ιο*3θ 010*31
6
LC70 cn\ c
12
c
c
13
74
011*00 011*01
011*10 011*11
0
a\ob\\
c J
15
012*10
012*11
011*20
011*21
012*20
012*21
011*30
011*31
012*30
012*31.
If, for example, we wish to express C74 as a product a,jb i, we
have to obtain the mixed radix representations of 7 and 4 with
respect to the bases [2, 4] and [ 3 , 2], respectively. Clearly,
k
7 = /.4 + * = 1.4 + 3 ;
Μ = M„_i ® · · · ® M i ® M
is unambiguously written without parentheses by the asso­
ciative law (14). From now on, we shall represent (20) by
4 = j2 + / = 2.2 + 0.
Hence, / = 1, k = 3 , 7 = 2, / = 0, and C74 = α 12*30·
The main properties of the Kronecker product of matrices
are gathered in the following theorem.
Theorem 2: Consider the matrices M M\, Mo, N\,No over
B. The following properties hold true whenever the corre­
sponding operations are defined:
1) (Associativity of the Kronecker product):
2t
M=
2
2) (Distributivity of the Kronecker product with respect
to the addition of matrices):
M.
(21)
k
Let us represent by m(i,j) and by m(i j )
the (ij) entry of
Μ and the (i J )
entry of M . Let us furthermore assume that
M is an (r , Q ) - m a t r i x . It is then easy to show that
kt k
k
k
k
k
k
n-\
(14)
2
®
k= n-\
m(i,j)
M ® ( M , ® Mo) = ( M ® M,)<g> A / Q .
(20)
0
= Π
k=0
(22)
rn (i ,j )
k
k
k
where / and j have the unique representations [i -u
, ^'i,
io] and \j„-\
· , 7 * 1 , 7 0 ] in the mixed radix systems with basis
vectors [r -\,
· , / * ! , ro] and [c«-i,
, c i , Co], respec­
tively.
Finally, the Kronecker product Μ ® Μ ®
® Μ of η
identical factors is called nth Kronecker power of Μ and is
denoted by Λ / Μ . Observe that
n
9
n
Mo ® (Ν + No) = (M ® Wi) + (Λ/ο β 7V )
λ
0
0
(15)
(Mi + M ) ® 7V = ( Μ ι β 7V ) + ( M Θ JV ). (16)
0
0
0
0
0
3) (Relationship between the ordinary and Kronecker
products of matrices):
(M1 <8> M )
0
4) If Τ denotes
(N\ β yV ) = (M,7V,) ® (M 7V ). (17)
0
0
transposition:
(Mi ® M ) = M j ® MJ.
T
0
5) / / M j
0
(18)
Mo 0 / * e invertible square matrices having the
1Λ«= ΐ ί .
We now turn to the extensions of Theorem 2 (3). These general
relationships between the ordinary and Kronecker matrix
products will play an important role in what follows.
Theorem 3: Whenever all the involved matrix products are
defined, the following properties hold true:
inverses Μ Γ and Mo :
1
(23)
Λΐ
-1
®
(Μ, ® M )
0
_ 1
= Μ Γ ® Mo"
1
(19)
k =n-
1
M
K
®
M=n—\
N )=
k
1
® '
k=n—\
(M N )
k
k
(24)
DAVIO: K R O N E C K E R
2)
Π
M
K
\k = n-\
J
0
3)
/
®
119
PRODUCTS
\ *
ft
Nk
\k = n-\
0
\
Π
Μ,/ =
A : = / 7 — 1 \l=n-\
J
=
J
0
Π
k=
/
(M ®N )
k
(25)
k
n-\
0
\
Π
®
Λ/,/.
l=n-\
\k=p-1
(26)
/
For the proof by recurrence of these properties, use (17) as
the starting point. The property (24) is apparently the only
published extension of (17). The other two properties will be
used in Section III.
3) Circuit diagram interpretation—Normal
factors: The
transformation w = Mg is easily interpreted in circuit terms:
an operator Μ seen as a black box acts upon an input vector
ν to produce an output vector vv. We shall prove in Section III
that if the transformation matrix Μ may be expressed as a
Kronecker product
the unique operator Μ splits up in
sets of smaller operators (black boxes) of the types M . Our
purpose is to describe accurately this decomposition process
and, in particular, the connection patterns relating the sets of
operators M . In the present section, we shall discover a hint
pointing to that decomposition. For this purpose, we consider
two particular cases
Fig. 1.
The concept of normal factor, (a) Μ = ( 1 ® MQ). (b) Μ = MQ
3
® 1 ).
3
M
Fig. 2.
k
k
0
The transformation w = M\Mog.
numberings identifies one of the copies of the operator MoClearly then, for i fixed the outputs labeled (/, 0), (/, 1), · · · ,
(/, c — 1) only depend on the inputs (/, 0)(/, 1),
, (/, r —
1). Therefore, the decomposition described by the normal
factor (\ ® Mo) has a simple interpretation in mixed radix
terms.
0
0
b
w=(U®A/o)g
and
vv^ = (MQ ®
\ )g
b
of the general transformation ^ = Mg. The circuit diagrams
corresponding to these two transformations are exemplified
in Fig. 1(a) and (b), respectively. W e assumed b = 3 and Mo
a 2 X 2 matrix. In both cases, the components of the input and
output vector are listed in lexicographic order.
It is now immediate to conclude that the transformation (1/,
® Mo) is represented, in a circuit diagram, by a set of b copies
of the operator M simply laid side by side. On the other hand,
the circuit diagram corresponding to (Mo ® h) contains a
similar arrangement of b copies of Mo, but the components of
the input and output vector have now to undergo an appro­
priate permutation. In Section 1I-C, we shall describe these
permutations as perfect shuffles. W e shall only observe now
that the Kronecker products of the type (l/> ® Mo) have a
particularly simple interpretation in terms of circuit diagrams.
This is the reason why these Kronecker products are normal
factors.
0
Consider now the transformation w = Mg and assume that
the transformation matrix Μ is the ordinary product M\M
of two matrices. In the corresponding circuit diagram, the
overall transformation will be represented by the cascade
connection of the operators M and M\. This is illustrated in
Fig. 2. Observe, to avoid any confusion, that Mo acts first on
the input vector, while M\, which appears as the leftmost factor
in M\M , only acts in a second transformation stage. Now we
immediately deduce that a Kronecker product of the type (Mo
® \ ) will admit an ordinary product representation: perfect
shuffle normal factor perfect shuffle.
We conclude this section by relating the concepts of normal
factor ( U ® Mo) and of mixed radix system. Assume that Mo
is an (r , c )-matrix. Then the inputs to the circuit (or equiv­
a l e n t ^ the components of the transformed vector v) may be
numbered in the mixed radix system with basis [b, r ]. Simi­
larly, the circuit outputs are numbered in the mixed radix
system with basis [b, c ]. The most significant digit of these
0
0
0
C. Perfect
Shuffles
1) Introductory example: Consider the problem of adding
two 4-bit numbers A = ( a , a , a\, ao) and Β = (b^ b%b\, bo)
initially stored in two 4-bit registers R and R\ by means of
a cascade of 4 full-adders. The corresponding circuit diagram
is shown in Fig. 3. T h e register outputs, the full-adders, and
the full-adders inputs have been systematically numbered as
indicated in the figure. The retained numbering has an inter­
esting property which allows one to describe accurately the
connection pattern relating the registers and the full-adders:
the ith output of register Rj is connected to the yth input of
full-adder FA/. This type of description is the essential feature
that we wish to retain in our definition of the perfect
shuffle.
2) Definition and elementary properties: Consider the set
L = (0, 1,
, m — 1} and assume that
3
2
0
m
m = b\bo.
(27)
The current element / may be represented in the mixed radix
systems having the basis vectors [b\, bo] and [bo, b\]. Let
/ = i\b
0
+ ill h, i* e \0, \,
+ io =
iuile
A (b\, bo)-shuffle
defined by
a(b\,
on L
m
b ):i
0
• , b - \ };
0
{0, 1,
- lj.
is a permutation a(b\, b ) of
= i\b
0
0
+ io
= iob\ + i\
(28)
L
m
(29)
b
0
0
0
0
where j = ίσ is the image of /. Thus, if / has the representation
[z'i, z'o], its image j = ζ'σ has the representation \j]jo] = [io,
i\]. The elements of L may obviously be partitioned in two
ways as follows.
m
7Ti) : b\ blocks of bo elements each. The e l e m e n t / = [i\,io]
will appear as element number / in block number i\\
7T ) : bo blocks of b\ elements each. The element j = [j],j*o]
will appear as element number jl in block number j * .
0
2
120
IEEE TRANSACTIONS ON COMPUTERS, VOL. C-30, NO. 2, FEBRUARY 1981
b [3,5]
15,3]
i [i i 1
j U?.fii
a
v
Fig. 3.
0
1
2
3
U
5
6
7
8
9
10
11
12
13
U
Addition of two 4-bit numbers.
0
0 0
01
02
0 3
0 U
1 0
11
1 2
1 3
1L
2 0
21
22
2 3
2L
0
1
2
3
U
5
6
7
8
9
10
11
12
13
U
Fig. 4.
The definition of the perfect shuffle now simply states that
the image of element io of block / 1 in π\ is the element i\ of
block io in π . Fig. 4 illustrates a (3, 5)-shuffle. Similarly, the
shuffle appearing in Fig. 3 is a (2,4)-shuffle while the shuffles
appearing in Fig. 1 (b) are from left to right a (2, 3)-shuffle and
a (3, 2)-shuffle. It is immediate to observe that
2
a(b b )
u
= o-\bo,b ).
0
(30)
x
Α σ(3, 5)-shuffle.
3) Matrix formulation:
U p to now, the shuffle has been
defined as a permutation, i.e., as a mapping on sets. To keep
consistent with our general discussion formalism, we now in­
troduce a matrix representation of the shuffles. As any per­
mutation, a (b\, *o)-shuffle may be represented by its adja­
cency matrix Sb\M- This matrix is a square matrix of order
b\bo whose rows and columns are numbered from 0 to b\bo —
1. W e denote by Sb b (j> 0
element of the matrix belonging
to row j and column /. T h e adjacency matrix is then defined
by
t
The above definition may now be applied in the broader
context of an arbitrary mixed radix system. Let
m = b -\
b bo.
(31)
As in Section II-A, we shall represent by w - \ the product b -i
b\
bo. Consider now the mixed radix representation
Un-u
' ' J\Jo] of an integer ζ (0 ^ / ^ m — 1) with respect
to the basis Vector [b -\, * ' · , *ι, *o]- Then,
Theorem 4: The mixed radix representation of
ia(b -\,
H > / I - I ) with respect to the basis vector [b -2,
· ,
bo, b -\]
n
u
x
n
iffy =
Sb b (j> 0 ~ 0
otherwise.
h
0
(32)
For example, the adjacency matrix £ 3 , 5 associated to the ( 3 ,
5)-shuffle of Fig. 4 is given by
"1
n
•••
e
SbuboU, 0 = ι
1
n
n
n
n
0
n
n
is [i -2,
0 0
01
0 2
1 0
11
1 2
2 0
21
22
30
31
32
U0
L 1
4 2
,i\,io,i -\]'
1
n
Proof: It suffices to observe that the mixed radix repre­
sentation of / with respect to the basis vector [b -\,
w -\]
is
n
1
1
n
^3,5
=
n-2
i -\,
n
Σ
ik^k
k=0
Hence, the representation of ia(b -\,
the basis vector [w -\, b -\] is
n
n
w-)
n
x
with respect to
n
n-2
Σ
iWk,
1
in-
k=0
1
i.e.,
ia(b -
w„-i) =
n u
|j£
ikWkj
°n-\
blank entries represent zeroes. Thus, in a transformation w =
one has, for instance, w = T J . As we already observed,
+ ι -ι.
Λ
Q.E.D.
S35V,
6
to the mapping τ = σ ^ : /
y = / σ ι 0 2 is associated the ma­
2
trix representation w - S2 · S\ · ν: the factors S2 and S \ of the
matrix product appear in reverse order as the permutation σ
is the first one to act on the element /. This should not cause
any confusion.
Before discussing additional properties of the shuffles and
of their adjacency matrices, we shall first interpret in terms
of mixed radix systems some important permutation matrices
which are built up from shuffle-matrices and Kronecker
products. These matrices will play an important role in the
sequel.
χ
Observe that the action of a(b -\, w -\) operates a cyclic
shift of one position to the left both on the mixed radix repre­
sentation of / on the basis vector b. T h e property is clearly of
particular interest when m = b and when / is given by its radix
b representation. In that situation [20], the radix b represen­
tation of ia(b, b ~ ) and of ia(b ~ ,
b) are deduced from the
radix b representation of / by one position left and right cyclic
shifts, respectively.
n
n
n
n
l
n
]
DAVIO: K R O N E C K E R
PRODUCTS
121
1) Consider first the permutation matrix
\
b
2
®S
b
h
b
b2b\,bo
b2bo,b\
>U\,
s
(33)
o
acting on a set of m = b2b\bo points. It represents a set of b
independent (bu /3o)-shuffles acting separately inside blocks
of b\bo points: this is a particular case of normal factor, as
described in Section II-B3. Computing the (/', /)-entry of the
matrix (33), one easily checks that if an input point i is given
by its representation [i , iu io] with respect to the basis [b ,
b\, bo], it will be mapped onto the output point j represented
as [ / 2 , io, M] in the [b , bo,b\] system. In particular, the ma­
trix
2
2
[ ' 2 , iu io]
s
>[io> ii, h]
'o, ' 2 ]
2) Similarly,
b2b\,bo
s
>[i , ii,
[ii, iu io]d
b b\,b
i
2
[12,
b ,bo* b\
9S
2
s
0
iu io]
i\]
0
l
2
> [h, «ο, i\]
• [h, h, i\]
2
\l - ®S k-
(34)
n k]
b
Kb
operates a cyclic shift of one position to the right on the k less
significant digits of the radix b representation of any domain
point.
2) Similarly, in
Sb M
® U
2
(35)
0
b b\ blocks of bo points each are submitted to a shuffle of
blocks, the position of the points within the blocks remaining
unchanged by the transformation. In particular, the matrix
2
operates a cyclic shift of one position to the right on the k most
significant digits of the radix b representation of any domain
point i. The matrices (33) and (35) are illustrated in the
right-hand part of Fig. 5(b).
3) Consider finally the permutation matrix
l f "
H
1
e
((h®S -i)
S )
bM
(37)
bKb
acting on the η digits of the radix b representation [ i „ - i ,
' ik+\,ik, ' ' ,i\, h]- The (k + 1) less significant digits first
undergo a cyclic right shift due to S k . The k less significant
digits next undergo a cyclic left shift due to S k-\.
Clearly,
the permutation described by (37) acts as the transposition of
digits (/*, i'o) in the radix b representation of i.
The interpretation in mixed radix terms of permutation
matrices of the types (33), (35), and (37) illustrates an inter­
esting proof methodology. A first application of this method­
ology will be found in the following Theorem 5 which presents
two factorizations of shuffles.
Theorem 5:
b
b
Q.E.D.
The interpretation of the above two properties is of interest.
Property (38) states that it is possible to decompose a perfect
shuffle acting on a small number of large size input blocks as
a cascade of two shuffles acting on a large number of small size
input blocks. Property (39) is even more striking since it de­
composes the shuffle S
into smaller shuffles S
and
Sb2.bo> ^ e former acting inside small blocks and the latter
acting as a shuffle of blocks. Those properties are illustrated
in Fig. 5 for b = 2, b\ = 2, b = 3.
4) Kronecker products and perfect shuffles: So far, we have
studied the relationship existing between perfect shuffles and
number representation systems and we used that relation to
enrich our repertory of shuffle properties. We shall now as­
certain the relation existing between Kronecker products and
perfect shuffles. Such a relationship had been demonstrated
informally in Section II-B3 in the study of (Mo ® \ ) and
motivated our study of shuffles. As the Kronecker product is
not commutative, we first study the relationship existing be­
tween M i ® Mo and Mo ® M\.
Theorem 6: If M\ and Mo are (r\, c\)- and ( r , Co)-matri­
ces, respectively,
then,
b l b h b o
2
bhbo
0
b
0
Mi ® M = S ,
0
r o
r i
· ( M ® Mi) · S .
0
(40)
ChCQ
btb
1) S
b 2 > b l b o
= S
2) S
b l b X i b ( i
= (S
•S
b l b 0 t b l
b2>bo
b l b x b o
= Sb b\M
2
' Sbib&bv
® Ui)(U ® Sb\,b ).
2
0
(38)
(39)
Proof: Let [i , iu io] be the representation of i with re­
spect to the basis vector [b , bubo]* W e study the transfor­
mations undergone by [i , iu io] in the various operations. The
bookkeeping of the basis vector is unnecessary since its evo­
lution parallels that of [i , i\ io].
1) We only prove the first equality: the second equality is
obtained by interchanging b\ and bo.
2
2
2
2
RON-L coc\-\
Σ
Σ
k=0
/=0
s , (i,k)P(k,i)s (ij)
ro n
(4i)
cuco
reduces to a single term. Indeed, there is a single value of k,
call it k, such that
SronO,
k) =
1.
By (32), this particular k is given by / = ka{r , r ), i.e., k =
iv(ru ro) = ' V i + i\. Similarly, there is a single value of /, call
it I, such that
0
x
9
b2,b\bo
S
[ii, iu «ο]
Proof: We compute the (i,y)-entry of the matrices in both
members of (40). If 1 = i\ro + io and j = j\Co + jo, the
entry of the left-hand member is m i ( / i , j\)mo(io, jo). To
perform the same computation in the right-hand member, we
denote by Ρ the matrix Mo ® M j . Then the ( / , g e n t r y of the
right-hand member, represented by the sum
• [iu io, h]
ScucoOj)
=
1.
Again, by (32) I = ja(cuc )
= joC\ + ju Finally, by (12) the
only nonzero term P{k, 1) in (41) is equal to
mo(ioJo)rn\(i\,
ji).
Q.E.D.
0
122
IEEE T R A N S A C T I O N S
ON COMPUTERS,
V O L . C-30,
N O . 2, F E B R U A R Y
1981
of n(ri, Ci)-matrices
M,. Our purpose is to represent the
Kronecker product ( 4 5 ) as the ordinary matrix product of η
factors of compatible sizes. In fact, the factorization Theorem
8 exhibits n\ such representations. Let χ represent a permu­
tation of {0, 1, · · · , η — 1}. Then,
Theorem 8:
ο
^2,6
J
6.2
J
®
6.2
0
Μι =
.Π,
0
<*;+1
J=iX
(46)
where
if ίχ > k\\
a
l
'2» 2,3
^2,3
J
4 3
Fig. 5.
w
'2
Factorization of perfect shuffles (Theorem 5).
As an immediate consequence, we obtain the following.
Theorem 7: If Μ is an (r\,
C\)-matrix
1) Μ Θ \
2) \
b l
b o
= S
®Μ ® U
3) U ® M ® \
2
= (\
• (\
b 0 t n
0
bo
®Μ)·
(42)
ChbQ
= S ^ r i • (Wo
^)
Θ
' b c\M
S
2
(43)
b o
QS )(l
b2
S
b0tri
b2b0
® M)(\
®S
b2
f
(44)
Proof: 1) is obtained from ( 4 0 ) by replacing in ( 4 0 ) Mo
by \ . 2) is obtained from ( 4 2 ) by replacing in ( 4 2 ) Μ by the
(bi^u *2^i)-matrix (\
® M) and by using the obvious
property \ ® \ = U /, . 3) by ( 4 2 ) , we also obtain
b o
bl
b l
l
b2
® (Μ
b o
2
0
® U ) = U ® (^
3
0
2
0>ri
·
(\
® M) ·
b0
S ).
cub0
The proof is then completed by using Theorem 3 - 2 ) .
b l
b o
b l
b o
FACTORIZATION OF KRONECKER PRODUCTS
Factorization
In the present
product
l
b
(47)
in such a way that M appears in position ίχ of that product
(positions are numbered from right to left). T h e proof is then
immediately completed by Theorem 3-3) and by the following
observation: assume k > /; then, in ( 4 0 ) , Mi will be multiplied
on the left (in the Kronecker sense) by a factor 1 ^ . Clearly,
this factor is \ if kx < ίχ and \ in the opposite situation.
A similar observation holds true if k < i.
Q.E.D.
Examples: a) For η = 2 , Theorem 8 yields the two factori­
zations
t
r k
C k
Μ ι ® Mo = ( M j ® l ) ( l
r o
c l
®M )
0
=
A. Fundamental
k
Λ-1-l'X
Q.E.D.
The property ( 4 2 ) was in fact our starting hint already il­
lustrated in Fig. 1. Observe also that the right-hand members
of ( 4 3 ) and ( 4 4 ) only contain shuffles and normal factors as
product factors. The consequence of this observation is that
we now have accurate circuit descriptions of \ ® Μ ® \ ,
including the input and output connection patterns. Essentially,
we should remember that the transformation \ ® Μ ® \
may be viewed in circuit terms as a set of b^bo operators per­
forming the transformation Μ preceded and followed by
perfect shuffles.
III.
= c if / χ < &χ.
Comment: Let us first interpret ( 4 6 ) . Observe that it re­
places the Kronecker product ( 4 5 ) by an ordinary product of
η factors having the form \ ® M ^ ® \ >\ by Theorem 7 we
shall be able to exhibit the corresponding normal factors and
shuffles. Remember now the concluding comment of the
preceding section: it shows that the transformation described
by ( 4 5 ) and performed as indicated by the right-hand member
of ( 4 6 ) may be viewed in circuit terms as the cascade connec­
tion of η stages; the yth stage only contains copies of the op­
erator performing the transformation Mj. Appropriate per­
mutations will relate the outputs of stage j to the inputs of stage
(/ + 1). The use of the permutation χ allows one to vary at will
the order of the stages. T h a t flexibility results of the relative
independence of the factors in a Kronecker product. Indeed,
the operators corresponding to the factor Λ/,· in ( 4 5 ) act on sets
of A*/ elements having all their coordinates identical but the /th.
The need for interposed shuffles is then justified by the fact
that normal factors only act on the least significant coordinate:
the shifting property of shuffles is used to bring in turn each
of the η coordinates in the least significant position.
Proof: Start from the left-hand member of ( 4 6 ) . Replace
each Kronecker factor Mi by the equivalent ordinary matrix
product of η factors
b
S
k
(l^MoKM!
For example, if
Theorem
section, we consider
M„-i ® · · · ® Mi ® M
0
the Kronecker
(45)
a b
Mi =
c d
ef
and Mo =
α β y
IJ.
(48)
DAVIO: KRONECKER PRODUCTS
123
then
\b
M\ ® Mo
:
"α β
δ €
Ύ!
Il­
TABLE Ι
FACTORIZATIONS O F M ® M\ ® M
2
X
IA
2
0 7
2
2
fj
7|
f1
\a
β
17
ι /
0
1
(Λ/ «
2
Λ
® ··® \
I
(· · · ® \
Γ
1
Γ
0
) ( 1 , ® Λ/,
0
2
0
1
2
®
M\
®
0
1
Λ/ΐ®
2
1
0
2 ( W ®M )(M ®
d r
®
2
0
)(1,
2
Γ
0
® Λ/,
β
0
2
0
ICQ)
®
2
2
-S .2 ( 1 β Α/ο)
2
2
= (U*M )-S -(UG
0
Λ/ )
1, )(Λ/ ®
= S2.3- ( 1 ® Μ ι )
0
ί
2
2
2
Π
® Λ/1 ®
1/·κο)0<·2
lr )(lr R, ® Λ / ) ( Λ /
2
0
2
Μι ® Uo)
C2
L R , r o ) ( L R R , <δ
lro)(A/ ®
8>Μο)(1
0
Γ
ΙΓ,,-ΟΜΙ^Ι ® M )(\ ®
2
0
)Λ/,)
3t3
Our present purpose is however not to give explicit de­
scriptions of all the possible circuits, but merely to concentrate
on some particular cases and to relate these circuits to previ­
ously described ones. From now on, we restrict ourselves to the
particular case where all the M/'s are square matrices of order
bj and where χ is the identity mapping on JO, 1, · , η — lj.
Under these assumptions, formula (46) may be restated as
®
M,-=
(1* -,-Α/ ι^/βΐ6Η-Αο).
ft
i=n—
Λ
(49)
+
1
To shorten the notations, we also define
® · · · ® Mi ®
Ck
1
2
/'=/?— 1
® · · ·)
n
«
2
The corresponding circuit diagrams are shown in Fig. 6(a) and
k
)
This rule may be called a quasi-commutative
rule: it indeed
reduces to the usual commutativity if all the matrices Μ,· are
square matrices (of any order).
2) Theorem 9 generalizes to an arbitrary Kronecker
product a factorization of the nth Kronecker power of a square
matrix used by Lechner [6], [10].
B. Circuit
( Λ /
Mi ® M
( ··® M
k
0
0
® · · · ® Mi ® · ·)
®M
1
1
y
b) The six factorizations corresponding to η — 3 are given
in Table I.
Remarks: 1) The ΑΪ! factorizations described by Theorem
8 are actually distinct, since they may in particular involve
matrices of different sizes. They may, however, be derived from
each other by applying step by step a simple rule, illustrated
here for i <k.
r k
0
Mi ® M
*
® \
M ® M\ ® Mo
1
1
yV
α β
0
(50)
W e now have
Theorem 9:
Mi=
Application
i—n—
It now only remains to make a joint use of the results ob­
tained in Sections II-C and III-A to complete the description
of a variety of circuits performing a product Mv when Μ has
a Kronecker product structure. It is immediate to observe that
all the factors appearing in the right-hand side of (46) are of
the type studied in Theorem 7-2) and 3). Accordingly, in the
most general situation, we shall be able to exhibit roughly
2
n\ distinct circuits performing the product Mv when Μ
is of the form (45): indeed, in each of the n\ factorizations (46)
we may replace each of the η factors by either one of its ex­
pressions (43) or (44).
Let us first illustrate our design philosophy by a simple ex­
ample. Consider the product Μ\ ® M , where M\ is a (2,
3)-matrix and Mo is a (3, 2)-matrix. By (48), we obtain
1
Π
[S ,
(\ ®Mi)].
Bl bi
(51)
Bi
i=n—\
Proof: By replacing in (49) each of the right-hand side
factors by its expression (43), one first obtains
ο
®
ο
Π
M/=
i=n—\
[Sbi-\---b .b„-i---bi
0
i=n—\
• (\
Bi
® Mi)S _
bn
_
x
bubi
].
x
bo
n
0
Λ/ι ® M
0
= (Μ, ® 1 )
2
(1 ®M )
2
0
In the latter expression, the factors ( 1 ^ . ® M/) and (1^,_, ®
M / _ i ) are separated by the product of shuffles
Sb -\
n
• • • bi,bj-\ - bo' Sb,- • • • bo,b -\ • • • b,-\2
An immediate computation making use of (38) finally shows
that the latter product of shuffles is actually equal to
^,-,Λ-i.
Q.E.D.
The philosophy underlying Theorem 9 is quite simple. Given
the initial representation [i -u ' ' ' , i i> *'o] of an input point
in the basis [/3 -ι, · · · , b\, bo], one replaces the action of the
Kronecker product appearing in the left-hand member of (51)
by the consecutive actions of a series of normal factors (1 ^ ®
M/), each of which acts on blocks of points differing only by
n
= ( 1 ® M ) ( M ! ® 1 ).
3
0
3
Let us now apply (42) to M\ ® 1 and to M i ® 1 , respectively.
We obtain
2
3
n
Λ
124
IEEE T R A N S A C T I O N S O N C O M P U T E R S , VOL. C-30, N O . 2, FEBRUARY
MQ
Mi
M2
MQ
Mi
M
M
Mi
Mz
MQ
Mi
M
Mo
M,
Mj
Mo
Mi
M
Mo
Mi
Mj
Mo
Mi
M
0
Fig. 6.
(b)
Circuit diagram for Μ\ ® MQ. (a) (Ι2 ® MQ); 5*2,2; (I2 ®
5 . 3 . (b) 5 , ; ( I 3 ® Μi); S , ; ( 1 ® Λ/ο).
2
2
3
3
3
Fig. 7.
MM
b
· (U»-i ® M)]
will easily be recognized as Good's factorization [2].
An equivalent circuit is obtained if one uses (44) instead of
(43) to obtain a circuit description of the Kronecker product
operation. We shall only illustrate the highlights of the cor­
responding interpretation in the case where all the matrices
Mi have the same order b. In this case, formula (49) may be
restated as
i—n—
Mi=
ft
1
i=n—
( 1 { Γ ' - ® Μ , · . ® 1 J'"!).
1 ]
(53)
1
If one replaces in (53) each factor of the right-hand side by its
expression (44), one obtains
®
i=n—\
M,= ft [ U l T ' - ® ^ )
1 1
Xd^-.^M/Kli"-'- !®^)].
1
Consider in the latter expression the factors inserted between
(U/i-i ® Mj+\) and (\ n-\ ® Mi). These factors are
b
(U' - /
2 1
0S^ i)(U' " ,
/
Circuit diagrams for Mi ® M\ ® MQ.
n
2
M
®Μι®M
I 1
+
The latter product may be written as
®S6'.6).
0
= 515.2(1 is ® Α/ )5Ίο.3
0
2
X (l,o® Mi)S , (l ®
6
5
6
M ).
0
2) For η = 2 (b = 2), one obtains, similarly,
3
M
2
® Mi ® M
0
= .S4.2O4
0
^2)^4,2
X ( 1 ® A / | ) S 4 . 2 ( l 4 ® Af ).
4
0
This circuit is illustrated by Fig. 7(a). The corresponding
handling by (53) and (54) yields
M
2
® Mi ® M
0
= £4,2(14 ® ^2)^2,4
X ( h ® S22XI4 ®
MDO2 ® S 2 2 X I 4
® Λ/ο).
This circuit is illustrated by Fig. 7(b).
IV.
i—n— 1
,
2
The latter expression is interpreted as (37). In this approach,
the consecutive digits i - 1 , · · , i\, io of the representation of
a domain point i are lead in the unit position by a sequence of
transpositions of the type (/*, io). The interconnection pattern
obviously changes from stage to stage and will easily be rec­
ognized to belong to the familiar Cooley-Tukey type [3].
Examples: 1) For b = 2, b\ = 3, b = 5, formula (51)
yields
2
®
2
2 ]
(52)
n
th
2
il""'- ®[V'-(U®%)].
r
= [S n-i
2
i);
3
the ith digit of their initial representations. These digits are
in turn lead to the unit position by consecutive cyclic shifts
caused by the shuffles Ss b
Observe, furthermore, that if all the matrices Μ,· are of order
b, the interstage connection pattern now described by Sb"-\b
is constant throughout the circuit. In particular, if all the
matrices Μ,· are identical, formula (51) written as
if
1981
CONCLUSIONS
The essential result obtained in this paper is probably the
discovery of a simple algebra able to describe a wide class of
switching circuits, ranging from discrete transform circuits
to permutation networks. The most interesting property of the
described algebra is probably the ease with which one may
derive families of equivalent circuits. It is, for example, in­
teresting to know that the constant geometry patterns, dis-
DAVIO: K R O N E C K E R
PRODUCTS
covered by Good in the context of H a d a m a r d transform, are
also applicable to permutation networks in the W a k s m a n
style.
From a theoretical point of view, we tried to use as few tools
as possible and managed to describe the main families of
practical circuits using only Kronecker products and perfect
shuffles. T h e complexity of (37) and (54), which basically
describe an elementary concept (transposition), suggests
however, an enrichment of our set of tools. This would probably
provide us with more concise and more sensible circuit de­
scriptions.
ACKNOWLEDGMENT
T h e author is indebted to Miss C. Gossart for fruitful dis­
cussions. H e also wishes to thank the referees of the paper for
extremely detailed and fruitful comments and criticisms.
REFERENCES
[1] C. K . Rushford, "Fast Fourier Hadamard decoding of orthogonal
codes," Inform. Contr., vol. 15, pp. 33-47, 1969.
[2] I. J. Good, "The interaction algorithm and practical Fourier analysis,"
J. Roy. Stat. Soc, ser. B, vol. 20, pp. 361-372,1958.
[3] J. W. Cooley and J. W. Tukey, "An algorithm for the machine com­
putation of complex Fourier," Series. Math. Comput., vol. 19, pp.
297-301, 1965.
[4] M. C. Pease, "An adaptation of the fast Fourier transform for parallel
processing," J. Ass. Comput. Mach., vol. 15, pp. 252-264, 1968.
[5] I. J. Good, "The relationship between two fast Fourier transforms,"
IEEE Trans. Comput., vol. C-20, pp. 310-317, 1971.
[6] R. J. Lechner, "Harmonic analysis of switching functions," in Recent
Developments in Switching Theory, A. Mukhopadhyay, Ed. New
York: Academic, 1971.
[7] A. Waksman, "A permutation network," J. Ass. Comput. Mach., vol.
15, pp. 159-163, 1968.
[8] H. S. Stone, "Parallel processing with the perfect shuffle," IEEE Trans.
Comput., vol. C-20, pp. 153-161, 1971.
[9] J. Lenfant, "Parallel permutations of data: A Benes network control
algorithm for frequently used permutations," IEEE Trans. Comput.,
vol.C-27, pp. 637-647, 1978.
[10] R.J. Lechner, "Transformations among switching function canonical
forms," IEEE Trans. Electron. Comput., vol. EC-12, no. 2, pp. 129-130,
1963.
125
[11] M. Davio, J. P. Deschamps, and A. Thayse, Discrete and Switching
Functions. New York: McGraw-Hill, 1978.
[12] B. J. Fino and R. Algazi, "A unified treatment of discrete fast unitary
transforms," SI AM J. Comput., vol. 6, pp. 700-717, 1977.
[13] E. O. Brigham, The Fast Fourier Transform. Englewood Cliffs, NJ:
Prentice-Hall, 1974.
[14] F. Yates, The Design and Analysis of Factorial Experiments. Harpenden: Imperial Bureau of Soil Science, 1937.
[15] D. E. Knuth, The Art of Computer Programming. Vol. 2: Seminumerical Algorithms. Reading, MA: Addison-Wesley, 1969.
[16] M. Davio and J. P. Deschamps, "Addition in signed digit number sys­
tems," in Proc. 8th Int. Conf. on Multiple Valued Logics, Chicago, IL,
1978, pp. 104-113.
[17] Ν. S. Szabo and R. J. Tanaka, Residue Arithmetic and its Applications
to Computer Technology. New York: McGraw-Hill, 1967.
[18] R. Bellman, Introduction to Matrix Analysis. New York: McGrawHill, 1960.
[19] S. W. Golomb, "Permutations by cutting and shuffling," SI AM Rev.,
vol.3, pp. 293-297, 1961.
[20] J. B. Kam and G. I. Davida, "Structured design of substitution-per­
mutation encryption networks," IEEE Trans. Comput., vol. C-28, pp.
747-753, Oct. 1979.
[21] S. B. Morris and R. E. Hartwig, "The generalized Faro shuffle," Discrete
Math., vol. 15, pp. 333-346, 1976.
[22] I. Cahit, "Realization of graceful permutations by a shuffle-exchange
network," Informat. Process. Lett., vol. 6, pp. 171-173, 1977.
Marc Davio was born in Monceau-sur-Sambre,
Belgium, in June 1938. He received the M.Sc. and
Ph.D. degrees in electrical engineering from the
University of Louvain, Louvain, Belgium, in 1961
and 1968, respectively.
He is currently a Research Group Leader at the
Philips Research Laboratory, Brussels, and Pro­
fessor at the University of Louvain, Louvain-laNeuve. He is engaged in research on digital circuit
design and algorithm implementation.