Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
116 I E E E T R A N S A C T I O N S O N C O M P U T E R S , V O L . C-30, N O . 2, F E B R U A R Y 1981 Kronecker Products and Shuffle Algebra MARC DAVIO Abstract—The paper relates three classical concepts, viz. mixed radix number system, Kronecker product of matrices, and perfect shuffle. It presents an algebra which describes the hardware organi zation of the computation of a product Mv, where Μ is a matrix in Kronecker product form and ν is a vector. The algebraic formalism describes both the blockwise structure of the computation and the various possible connection patterns. Index Terms—Kronecker product of matrices, mixed radix number systems, perfect shuffle. C I. INTRODUCTION E L L U L A R logic has, for a long time, attracted the at tention of designers. In the recent past, arrays of logical operators showing up a regular connection pattern, such as arithmetic arrays, have raised up a renewed interest in the perspective of their implementation in LSI form. The present paper addresses a class of circuits whose aim is to compute the transform w = Mv of a vector £ by a matrix Μ when the latter has a Kronecker product (or direct product) structure. The interest of this structure is that it corresponds to a decomposition of the circuit computing w: in many prac tical instances, the resulting circuit will consist of a regular interconnection of identical operators. Such a situation is encountered in quite a number of appli cation fields viz. H a d a m a r d transform [1], discrete Fourier transform [ 2 ] - [ 6 ] , connection and substitution networks [7]-[9], [20], canonical expansion of discrete functions [10], [11], • · · . It is only possible to give here a small sample of the subject literature and we tried to select, among the relevant references, those whose formalism was the closest to the one we use here. For additional references, refer, for example, to [12] and [13]. It is interesting to recall here some of the milestones in the development of the underlying theory: 1) discovery of the constant geometry networks computing the H a d a m a r d transform (1958, [2]), 2) quantitative estimation of the advantages obtained from the Kronecker product structure, both in hardware investment and in storage requirement (1963, [10]), 3) description of the fast Fourier transform ( F F T ) as a product of transformation and of permutation matrices (1968, [4]), and 4) arithmetic description of the perfect shuffle establishing its relation with the binary number system (1971, [8]). Most often these results have been obtained in a specific context (say the discrete transform theory) and are little or not Manuscript received April 2,1979; revised March 10, 1980 and September 10, 1980. The author is with Philips Research Laboratory, Brussels, Belgium. known in other possible application contexts (say permutation networks). The main result of the present paper is to establish a common background for all these problems; it is achieved thanks to three basic concepts: the mixed radix number rep resentation systems, the Kronecker product of matrices, and the (generalized) perfect shuffle. While these concepts are classical, their tight relationships probably never have been clearly stated. This is the subject matter of Section II, which furthermore lays the basis of a shuffle algebra and establishes a set of relations describing commonly encountered connection patterns. Section III is devoted to the factorization of Kro necker products. It first presents a decomposition theorem, which extends to Kronecker products of arbitrary matrices a factorization of Kronecker powers of square matrices first described by Lechner [10]. It next uses the shuffle algebra to derive from the basic decomposition theorem a number of equivalent circuits. The interconnection patterns of the Cooley-Tukey [3] and of the Yates [14], Good [2] types appear as particular cases of the obtained circuits. II. B A S I C C O N C E P T S A. Mixed Radix Number Representation Systems 1) Introductory example: If we wish to convert into seconds a duration such as 3 days 7 h 22 min 4 s (written [3, 7, 22,4]), we use the formula 3.86400 + 7.3600 + 22.60 + 4 = 285724. The obtained number of seconds appear as a weighted sum of the duration [3, 7, 22, 4 ] . The weights are given by the vector [86400, 3600, 60, 1]. 2) Definition and representation theorem: This familiar situation is imbedded in the concept of mixed radix number representation system that we now define formally. Consider an integer valued vector b= [b - ,b b ]; n h u bi>2. 0 (1) This vector is called basis vector. Build, from this vector a second vector w = [w , w n • • •, w , w ] n h x 0 (2) called weight vector and defined by i-\ w = 1; Wi = Wi-xbi-χ 0 = Π bj\ is ( 1 , 2 , · · · , n}. (3) 7=0 Theorem 0018-9340/81/0200-0116500.75 1: Any integer A such that 0^A<w n © 1981 I E E E (4) DAVIO: K R O N E C K E R P R O D U C T S 117 has a unique representation two following conditions: [a -\, , a\, ao] satisfying n the (5) i=0 2) O^atKbi; \fie {0, 1, · · , η - 1}. (6) Theorem 1 is merely restated here for the sake of com pleteness and its proof will not be reproduced. It may be found in [15] or [16]. The vector [a -\, · · ·, a\, ao] is the mixed radix representation of A with respect to the basis \b -\, , bu bo\. If b b\ - · = b -\ = b, the corresponding rep resentation is called radix b representation. Outside of the present context, mixed radix representations are also used in residue arithmetic. See, for example, [17]. n n = 0 n B. Kronecker Product of Matrices 1) Introductory remarks: The H a d a m a r d transform h(v) of the vector ν = [vo, v\, υ2, is the vector [Ao, h\, h , Λ3] computed according to the familiar matrix product 2 ho 1 hi 1 -1 h 1 h 1 -1 3 1 1 1 1 1 -1 v\ •1 - 1 •1 1 L y (7) 3 or, in short, h = Hv. (8) The computation of h from ν according to (7) apparently requires four scalar products, each involving three additions. For vectors ν having 2 components, the computation of the Hadamard transform would require 2 scalar products, each involving 2 — 1 additions. The particular block structure of the transformation matrix, suggested by the dotted lines in (7), allows one to perform the computation according to a better suited algorithm, using no more than n.2 ~ additions or subtractions. When pro grammed on a digital computer, the algorithm only requires 2 cells of working storage [10]. The matrix structure taken to advantage in order to achieve the above improvements is that of Kronecker product of ma trices. Basically, the Kronecker product Μ Θ Ν of the two matrices n n n n x n M = a b ; N = A C Β = aB bA bB- aC aD bC bD cA cB dA dB •cC cD dC dD. Γ Xm 0 D •aA 1 .1-1 The addition and multiplication involved in the two con sidered types of matrix products (Kronecker and usual) are, in the case of the H a d a m a r d transform, the addition and multiplication of real numbers. A similar situation is observed however in other contexts involving other additive and multi plicative laws. For example, if one expresses the canonical disjunctive expansion of a Boolean function in Kronecker product form, the underlying operations will be the Boolean sum and product. Similarly, the canonical ring sum expansions of Boolean functions, sometimes known as the Reed-Muller expansions [11], will be expressed by means of the modulo two sum and product. Finally, the computation of the discrete Fourier transform is expressed with the addition and multi plication of complex numbers. To cover these various situa tions, we shall present the definition and properties of the Kronecker product in terms of an abstract algebraic struc ture. 2) Definition and properties of the Kronecker product: Consider an algebraic system (B, + , ) involving the set Β and two operations, the addition ( + ) and the multiplication ( or no symbol) satisfying the following axioms (V·*, y, z, e B). 1) (associativity) χ + (y + z) = (x + y) + z; x(yz) = (xy)z. 2) (commutativity) χ -I- y = y -μ χ; χγ = y 3) (distributivity) x(y + ζ) = xy + xz. 4) (neutral elements) 3 0 , 1 e B:0 + x = χ; l. χ = χ. Observe that our algebraic system could be a complete distributive lattice or a commutative ring with unit. A (r, c) matrix Μ over Β is a set of rc elements of Β disposed in r rows and c columns. The rows and columns of Μ are numbered from 0 to (r — 1) and from 0 to (c — 1), respectively. The matrix element belonging to row number i and to column number j is represented by m(i,j). A (b, 6)-matrix is a square matrix of order b. The axioms governing the algebraic system Β allow one to define in the usual way the addition of matrices ( + ) and the (usual) product M\ · M or M\M of an ( r ci)-matrix M\ by a ( r , c o ) - m a t r i x M . The addition of ma trices is both associative and commutative. The product of matrices is associative and distributive over the addition. Furthermore, one may define the unit matrix of order b, \ by 0 is given by M®N 2 4 n-\ 2 their storage will require 2p memory locations, while the storage of their Kronecker product would require p double precision memory locations. Similarly, when computing the matrix product h = Hv, we shall only perform smaller matrix products involving the Kronecker factors of H; these factors have the form 0 1? 0 b U(/.y)=l (9) and the key point to the understanding of the new algorithm to compute h will rest upon the relationship between the Kronecker product and the usual product of matrices. That property will appear as Theorem 3 in the next section. Observe that if Μ and Ν are two square ρ Χ ρ matrices, iff/ = 7 (10) = 0 iff/ ^ y and check that for any (r, c)-matrix Μ \ Μ = Μ \ = Μ. (11) All these facts are elementary consequences of our axioms. The Kronecker product Μ\ <g> M of an (r cj)-matrix M\ by an ( r , C o ) - m a t r i x M is an (r\r , c\c )-matrix Μ defined by r c 0 0 0 0 u 0 118 IEEE m(i,j) = m\U\J\)mo(i Jo); 0 i = iVo + io'J =j\Co in) + jo The row and column indexes of the Kronecker product Μ thus have the representations [i\Jo] and \j\Jo] in the mixed radix systems with basis vectors [r\, r ] and [c\, Co], respectively. The block structure of Μ is fairly apparent: /1 and j\ are the block coordinates, while /'o and j are the element coordinates 0 0 within the ( / 1 , 7 1 )-block. Consider as an example the Kronecker product C = A ® Β of a (2, 3)-matrix A by a (4, 2)-matrix Β ' boo bo\' A= 0oo 001 002 a\o a\\ ; 012, b\o bu B = bio bix •*30 *3I (13) -1 TRANSACTIONS ON COMPUTERS, V O L . C-30, N O . 2, FEBRUARY 1981 These properties are now classical and their proofs may be found in [11], [18]; they have been restated here for reference. The most important of these properties is probably contained in (17), which relates the ordinary and the Kronecker matrix products. That property contains the essence of the simplifying mechanism mentioned in Section II-B1. Basically, this mechanism may be described as follows: if, in the computation of the matrix product w = M.g, the transformation matrix Μ may be expressed as the Kronecker product M i ® Mo, then, it is possible to compute w from independent transformations described by the matrices M i and M ; (17), in fact, contains a decomposition process: the latter will be illustrated in Section III. Equation (17) may be extended in various ways that will be described in Theorem 3 below. Before stating this theorem, we need some additional notations. The Kronecker product 0 C is thus the (8, 6)-matrix COO C0\ C02 C03 C 4 C05 C\QC\\ C 1 2 C 1 3 C 1 4 C 1 5 000*00 000*01 001*00 001*01 002*00 000*10 000*11 001*10 001*11 002*10 002*11 C oC \ ^22 C C 4 C 25 000*20 000*21 001*20 001*21 002*20 002*21 ^30^31 C32 C33 C34 C35 000*30 000*31 001*30 001*31 002*30 002*31 012*00 012*01 0 2 2 23 2 002*01 C = C40 C41 C42 C43 C44 C45 0 10*00 0 1 *01 cso c \ csi C53 C54 C55 C60 Ce 1 C 2 <?63 C(>4 ^ 6 5 010*10 S 010*20 010*21 [_0ιο*3θ 010*31 6 LC70 cn\ c 12 c c 13 74 011*00 011*01 011*10 011*11 0 a\ob\\ c J 15 012*10 012*11 011*20 011*21 012*20 012*21 011*30 011*31 012*30 012*31. If, for example, we wish to express C74 as a product a,jb i, we have to obtain the mixed radix representations of 7 and 4 with respect to the bases [2, 4] and [ 3 , 2], respectively. Clearly, k 7 = /.4 + * = 1.4 + 3 ; Μ = M„_i ® · · · ® M i ® M is unambiguously written without parentheses by the asso ciative law (14). From now on, we shall represent (20) by 4 = j2 + / = 2.2 + 0. Hence, / = 1, k = 3 , 7 = 2, / = 0, and C74 = α 12*30· The main properties of the Kronecker product of matrices are gathered in the following theorem. Theorem 2: Consider the matrices M M\, Mo, N\,No over B. The following properties hold true whenever the corre sponding operations are defined: 1) (Associativity of the Kronecker product): 2t M= 2 2) (Distributivity of the Kronecker product with respect to the addition of matrices): M. (21) k Let us represent by m(i,j) and by m(i j ) the (ij) entry of Μ and the (i J ) entry of M . Let us furthermore assume that M is an (r , Q ) - m a t r i x . It is then easy to show that kt k k k k k k n-\ (14) 2 ® k= n-\ m(i,j) M ® ( M , ® Mo) = ( M ® M,)<g> A / Q . (20) 0 = Π k=0 (22) rn (i ,j ) k k k where / and j have the unique representations [i -u , ^'i, io] and \j„-\ · , 7 * 1 , 7 0 ] in the mixed radix systems with basis vectors [r -\, · , / * ! , ro] and [c«-i, , c i , Co], respec tively. Finally, the Kronecker product Μ ® Μ ® ® Μ of η identical factors is called nth Kronecker power of Μ and is denoted by Λ / Μ . Observe that n 9 n Mo ® (Ν + No) = (M ® Wi) + (Λ/ο β 7V ) λ 0 0 (15) (Mi + M ) ® 7V = ( Μ ι β 7V ) + ( M Θ JV ). (16) 0 0 0 0 0 3) (Relationship between the ordinary and Kronecker products of matrices): (M1 <8> M ) 0 4) If Τ denotes (N\ β yV ) = (M,7V,) ® (M 7V ). (17) 0 0 transposition: (Mi ® M ) = M j ® MJ. T 0 5) / / M j 0 (18) Mo 0 / * e invertible square matrices having the 1Λ«= ΐ ί . We now turn to the extensions of Theorem 2 (3). These general relationships between the ordinary and Kronecker matrix products will play an important role in what follows. Theorem 3: Whenever all the involved matrix products are defined, the following properties hold true: inverses Μ Γ and Mo : 1 (23) Λΐ -1 ® (Μ, ® M ) 0 _ 1 = Μ Γ ® Mo" 1 (19) k =n- 1 M K ® M=n—\ N )= k 1 ® ' k=n—\ (M N ) k k (24) DAVIO: K R O N E C K E R 2) Π M K \k = n-\ J 0 3) / ® 119 PRODUCTS \ * ft Nk \k = n-\ 0 \ Π Μ,/ = A : = / 7 — 1 \l=n-\ J = J 0 Π k= / (M ®N ) k (25) k n-\ 0 \ Π ® Λ/,/. l=n-\ \k=p-1 (26) / For the proof by recurrence of these properties, use (17) as the starting point. The property (24) is apparently the only published extension of (17). The other two properties will be used in Section III. 3) Circuit diagram interpretation—Normal factors: The transformation w = Mg is easily interpreted in circuit terms: an operator Μ seen as a black box acts upon an input vector ν to produce an output vector vv. We shall prove in Section III that if the transformation matrix Μ may be expressed as a Kronecker product the unique operator Μ splits up in sets of smaller operators (black boxes) of the types M . Our purpose is to describe accurately this decomposition process and, in particular, the connection patterns relating the sets of operators M . In the present section, we shall discover a hint pointing to that decomposition. For this purpose, we consider two particular cases Fig. 1. The concept of normal factor, (a) Μ = ( 1 ® MQ). (b) Μ = MQ 3 ® 1 ). 3 M Fig. 2. k k 0 The transformation w = M\Mog. numberings identifies one of the copies of the operator MoClearly then, for i fixed the outputs labeled (/, 0), (/, 1), · · · , (/, c — 1) only depend on the inputs (/, 0)(/, 1), , (/, r — 1). Therefore, the decomposition described by the normal factor (\ ® Mo) has a simple interpretation in mixed radix terms. 0 0 b w=(U®A/o)g and vv^ = (MQ ® \ )g b of the general transformation ^ = Mg. The circuit diagrams corresponding to these two transformations are exemplified in Fig. 1(a) and (b), respectively. W e assumed b = 3 and Mo a 2 X 2 matrix. In both cases, the components of the input and output vector are listed in lexicographic order. It is now immediate to conclude that the transformation (1/, ® Mo) is represented, in a circuit diagram, by a set of b copies of the operator M simply laid side by side. On the other hand, the circuit diagram corresponding to (Mo ® h) contains a similar arrangement of b copies of Mo, but the components of the input and output vector have now to undergo an appro priate permutation. In Section 1I-C, we shall describe these permutations as perfect shuffles. W e shall only observe now that the Kronecker products of the type (l/> ® Mo) have a particularly simple interpretation in terms of circuit diagrams. This is the reason why these Kronecker products are normal factors. 0 Consider now the transformation w = Mg and assume that the transformation matrix Μ is the ordinary product M\M of two matrices. In the corresponding circuit diagram, the overall transformation will be represented by the cascade connection of the operators M and M\. This is illustrated in Fig. 2. Observe, to avoid any confusion, that Mo acts first on the input vector, while M\, which appears as the leftmost factor in M\M , only acts in a second transformation stage. Now we immediately deduce that a Kronecker product of the type (Mo ® \ ) will admit an ordinary product representation: perfect shuffle normal factor perfect shuffle. We conclude this section by relating the concepts of normal factor ( U ® Mo) and of mixed radix system. Assume that Mo is an (r , c )-matrix. Then the inputs to the circuit (or equiv a l e n t ^ the components of the transformed vector v) may be numbered in the mixed radix system with basis [b, r ]. Simi larly, the circuit outputs are numbered in the mixed radix system with basis [b, c ]. The most significant digit of these 0 0 0 C. Perfect Shuffles 1) Introductory example: Consider the problem of adding two 4-bit numbers A = ( a , a , a\, ao) and Β = (b^ b%b\, bo) initially stored in two 4-bit registers R and R\ by means of a cascade of 4 full-adders. The corresponding circuit diagram is shown in Fig. 3. T h e register outputs, the full-adders, and the full-adders inputs have been systematically numbered as indicated in the figure. The retained numbering has an inter esting property which allows one to describe accurately the connection pattern relating the registers and the full-adders: the ith output of register Rj is connected to the yth input of full-adder FA/. This type of description is the essential feature that we wish to retain in our definition of the perfect shuffle. 2) Definition and elementary properties: Consider the set L = (0, 1, , m — 1} and assume that 3 2 0 m m = b\bo. (27) The current element / may be represented in the mixed radix systems having the basis vectors [b\, bo] and [bo, b\]. Let / = i\b 0 + ill h, i* e \0, \, + io = iuile A (b\, bo)-shuffle defined by a(b\, on L m b ):i 0 • , b - \ }; 0 {0, 1, - lj. is a permutation a(b\, b ) of = i\b 0 0 + io = iob\ + i\ (28) L m (29) b 0 0 0 0 where j = ίσ is the image of /. Thus, if / has the representation [z'i, z'o], its image j = ζ'σ has the representation \j]jo] = [io, i\]. The elements of L may obviously be partitioned in two ways as follows. m 7Ti) : b\ blocks of bo elements each. The e l e m e n t / = [i\,io] will appear as element number / in block number i\\ 7T ) : bo blocks of b\ elements each. The element j = [j],j*o] will appear as element number jl in block number j * . 0 2 120 IEEE TRANSACTIONS ON COMPUTERS, VOL. C-30, NO. 2, FEBRUARY 1981 b [3,5] 15,3] i [i i 1 j U?.fii a v Fig. 3. 0 1 2 3 U 5 6 7 8 9 10 11 12 13 U Addition of two 4-bit numbers. 0 0 0 01 02 0 3 0 U 1 0 11 1 2 1 3 1L 2 0 21 22 2 3 2L 0 1 2 3 U 5 6 7 8 9 10 11 12 13 U Fig. 4. The definition of the perfect shuffle now simply states that the image of element io of block / 1 in π\ is the element i\ of block io in π . Fig. 4 illustrates a (3, 5)-shuffle. Similarly, the shuffle appearing in Fig. 3 is a (2,4)-shuffle while the shuffles appearing in Fig. 1 (b) are from left to right a (2, 3)-shuffle and a (3, 2)-shuffle. It is immediate to observe that 2 a(b b ) u = o-\bo,b ). 0 (30) x Α σ(3, 5)-shuffle. 3) Matrix formulation: U p to now, the shuffle has been defined as a permutation, i.e., as a mapping on sets. To keep consistent with our general discussion formalism, we now in troduce a matrix representation of the shuffles. As any per mutation, a (b\, *o)-shuffle may be represented by its adja cency matrix Sb\M- This matrix is a square matrix of order b\bo whose rows and columns are numbered from 0 to b\bo — 1. W e denote by Sb b (j> 0 element of the matrix belonging to row j and column /. T h e adjacency matrix is then defined by t The above definition may now be applied in the broader context of an arbitrary mixed radix system. Let m = b -\ b bo. (31) As in Section II-A, we shall represent by w - \ the product b -i b\ bo. Consider now the mixed radix representation Un-u ' ' J\Jo] of an integer ζ (0 ^ / ^ m — 1) with respect to the basis Vector [b -\, * ' · , *ι, *o]- Then, Theorem 4: The mixed radix representation of ia(b -\, H > / I - I ) with respect to the basis vector [b -2, · , bo, b -\] n u x n iffy = Sb b (j> 0 ~ 0 otherwise. h 0 (32) For example, the adjacency matrix £ 3 , 5 associated to the ( 3 , 5)-shuffle of Fig. 4 is given by "1 n ••• e SbuboU, 0 = ι 1 n n n n 0 n n is [i -2, 0 0 01 0 2 1 0 11 1 2 2 0 21 22 30 31 32 U0 L 1 4 2 ,i\,io,i -\]' 1 n Proof: It suffices to observe that the mixed radix repre sentation of / with respect to the basis vector [b -\, w -\] is n 1 1 n ^3,5 = n-2 i -\, n Σ ik^k k=0 Hence, the representation of ia(b -\, the basis vector [w -\, b -\] is n n w-) n x with respect to n n-2 Σ iWk, 1 in- k=0 1 i.e., ia(b - w„-i) = n u |j£ ikWkj °n-\ blank entries represent zeroes. Thus, in a transformation w = one has, for instance, w = T J . As we already observed, + ι -ι. Λ Q.E.D. S35V, 6 to the mapping τ = σ ^ : / y = / σ ι 0 2 is associated the ma 2 trix representation w - S2 · S\ · ν: the factors S2 and S \ of the matrix product appear in reverse order as the permutation σ is the first one to act on the element /. This should not cause any confusion. Before discussing additional properties of the shuffles and of their adjacency matrices, we shall first interpret in terms of mixed radix systems some important permutation matrices which are built up from shuffle-matrices and Kronecker products. These matrices will play an important role in the sequel. χ Observe that the action of a(b -\, w -\) operates a cyclic shift of one position to the left both on the mixed radix repre sentation of / on the basis vector b. T h e property is clearly of particular interest when m = b and when / is given by its radix b representation. In that situation [20], the radix b represen tation of ia(b, b ~ ) and of ia(b ~ , b) are deduced from the radix b representation of / by one position left and right cyclic shifts, respectively. n n n n l n ] DAVIO: K R O N E C K E R PRODUCTS 121 1) Consider first the permutation matrix \ b 2 ®S b h b b2b\,bo b2bo,b\ >U\, s (33) o acting on a set of m = b2b\bo points. It represents a set of b independent (bu /3o)-shuffles acting separately inside blocks of b\bo points: this is a particular case of normal factor, as described in Section II-B3. Computing the (/', /)-entry of the matrix (33), one easily checks that if an input point i is given by its representation [i , iu io] with respect to the basis [b , b\, bo], it will be mapped onto the output point j represented as [ / 2 , io, M] in the [b , bo,b\] system. In particular, the ma trix 2 2 [ ' 2 , iu io] s >[io> ii, h] 'o, ' 2 ] 2) Similarly, b2b\,bo s >[i , ii, [ii, iu io]d b b\,b i 2 [12, b ,bo* b\ 9S 2 s 0 iu io] i\] 0 l 2 > [h, «ο, i\] • [h, h, i\] 2 \l - ®S k- (34) n k] b Kb operates a cyclic shift of one position to the right on the k less significant digits of the radix b representation of any domain point. 2) Similarly, in Sb M ® U 2 (35) 0 b b\ blocks of bo points each are submitted to a shuffle of blocks, the position of the points within the blocks remaining unchanged by the transformation. In particular, the matrix 2 operates a cyclic shift of one position to the right on the k most significant digits of the radix b representation of any domain point i. The matrices (33) and (35) are illustrated in the right-hand part of Fig. 5(b). 3) Consider finally the permutation matrix l f " H 1 e ((h®S -i) S ) bM (37) bKb acting on the η digits of the radix b representation [ i „ - i , ' ik+\,ik, ' ' ,i\, h]- The (k + 1) less significant digits first undergo a cyclic right shift due to S k . The k less significant digits next undergo a cyclic left shift due to S k-\. Clearly, the permutation described by (37) acts as the transposition of digits (/*, i'o) in the radix b representation of i. The interpretation in mixed radix terms of permutation matrices of the types (33), (35), and (37) illustrates an inter esting proof methodology. A first application of this method ology will be found in the following Theorem 5 which presents two factorizations of shuffles. Theorem 5: b b Q.E.D. The interpretation of the above two properties is of interest. Property (38) states that it is possible to decompose a perfect shuffle acting on a small number of large size input blocks as a cascade of two shuffles acting on a large number of small size input blocks. Property (39) is even more striking since it de composes the shuffle S into smaller shuffles S and Sb2.bo> ^ e former acting inside small blocks and the latter acting as a shuffle of blocks. Those properties are illustrated in Fig. 5 for b = 2, b\ = 2, b = 3. 4) Kronecker products and perfect shuffles: So far, we have studied the relationship existing between perfect shuffles and number representation systems and we used that relation to enrich our repertory of shuffle properties. We shall now as certain the relation existing between Kronecker products and perfect shuffles. Such a relationship had been demonstrated informally in Section II-B3 in the study of (Mo ® \ ) and motivated our study of shuffles. As the Kronecker product is not commutative, we first study the relationship existing be tween M i ® Mo and Mo ® M\. Theorem 6: If M\ and Mo are (r\, c\)- and ( r , Co)-matri ces, respectively, then, b l b h b o 2 bhbo 0 b 0 Mi ® M = S , 0 r o r i · ( M ® Mi) · S . 0 (40) ChCQ btb 1) S b 2 > b l b o = S 2) S b l b X i b ( i = (S •S b l b 0 t b l b2>bo b l b x b o = Sb b\M 2 ' Sbib&bv ® Ui)(U ® Sb\,b ). 2 0 (38) (39) Proof: Let [i , iu io] be the representation of i with re spect to the basis vector [b , bubo]* W e study the transfor mations undergone by [i , iu io] in the various operations. The bookkeeping of the basis vector is unnecessary since its evo lution parallels that of [i , i\ io]. 1) We only prove the first equality: the second equality is obtained by interchanging b\ and bo. 2 2 2 2 RON-L coc\-\ Σ Σ k=0 /=0 s , (i,k)P(k,i)s (ij) ro n (4i) cuco reduces to a single term. Indeed, there is a single value of k, call it k, such that SronO, k) = 1. By (32), this particular k is given by / = ka{r , r ), i.e., k = iv(ru ro) = ' V i + i\. Similarly, there is a single value of /, call it I, such that 0 x 9 b2,b\bo S [ii, iu «ο] Proof: We compute the (i,y)-entry of the matrices in both members of (40). If 1 = i\ro + io and j = j\Co + jo, the entry of the left-hand member is m i ( / i , j\)mo(io, jo). To perform the same computation in the right-hand member, we denote by Ρ the matrix Mo ® M j . Then the ( / , g e n t r y of the right-hand member, represented by the sum • [iu io, h] ScucoOj) = 1. Again, by (32) I = ja(cuc ) = joC\ + ju Finally, by (12) the only nonzero term P{k, 1) in (41) is equal to mo(ioJo)rn\(i\, ji). Q.E.D. 0 122 IEEE T R A N S A C T I O N S ON COMPUTERS, V O L . C-30, N O . 2, F E B R U A R Y 1981 of n(ri, Ci)-matrices M,. Our purpose is to represent the Kronecker product ( 4 5 ) as the ordinary matrix product of η factors of compatible sizes. In fact, the factorization Theorem 8 exhibits n\ such representations. Let χ represent a permu tation of {0, 1, · · · , η — 1}. Then, Theorem 8: ο ^2,6 J 6.2 J ® 6.2 0 Μι = .Π, 0 <*;+1 J=iX (46) where if ίχ > k\\ a l '2» 2,3 ^2,3 J 4 3 Fig. 5. w '2 Factorization of perfect shuffles (Theorem 5). As an immediate consequence, we obtain the following. Theorem 7: If Μ is an (r\, C\)-matrix 1) Μ Θ \ 2) \ b l b o = S ®Μ ® U 3) U ® M ® \ 2 = (\ • (\ b 0 t n 0 bo ®Μ)· (42) ChbQ = S ^ r i • (Wo ^) Θ ' b c\M S 2 (43) b o QS )(l b2 S b0tri b2b0 ® M)(\ ®S b2 f (44) Proof: 1) is obtained from ( 4 0 ) by replacing in ( 4 0 ) Mo by \ . 2) is obtained from ( 4 2 ) by replacing in ( 4 2 ) Μ by the (bi^u *2^i)-matrix (\ ® M) and by using the obvious property \ ® \ = U /, . 3) by ( 4 2 ) , we also obtain b o bl b l l b2 ® (Μ b o 2 0 ® U ) = U ® (^ 3 0 2 0>ri · (\ ® M) · b0 S ). cub0 The proof is then completed by using Theorem 3 - 2 ) . b l b o b l b o FACTORIZATION OF KRONECKER PRODUCTS Factorization In the present product l b (47) in such a way that M appears in position ίχ of that product (positions are numbered from right to left). T h e proof is then immediately completed by Theorem 3-3) and by the following observation: assume k > /; then, in ( 4 0 ) , Mi will be multiplied on the left (in the Kronecker sense) by a factor 1 ^ . Clearly, this factor is \ if kx < ίχ and \ in the opposite situation. A similar observation holds true if k < i. Q.E.D. Examples: a) For η = 2 , Theorem 8 yields the two factori zations t r k C k Μ ι ® Mo = ( M j ® l ) ( l r o c l ®M ) 0 = A. Fundamental k Λ-1-l'X Q.E.D. The property ( 4 2 ) was in fact our starting hint already il lustrated in Fig. 1. Observe also that the right-hand members of ( 4 3 ) and ( 4 4 ) only contain shuffles and normal factors as product factors. The consequence of this observation is that we now have accurate circuit descriptions of \ ® Μ ® \ , including the input and output connection patterns. Essentially, we should remember that the transformation \ ® Μ ® \ may be viewed in circuit terms as a set of b^bo operators per forming the transformation Μ preceded and followed by perfect shuffles. III. = c if / χ < &χ. Comment: Let us first interpret ( 4 6 ) . Observe that it re places the Kronecker product ( 4 5 ) by an ordinary product of η factors having the form \ ® M ^ ® \ >\ by Theorem 7 we shall be able to exhibit the corresponding normal factors and shuffles. Remember now the concluding comment of the preceding section: it shows that the transformation described by ( 4 5 ) and performed as indicated by the right-hand member of ( 4 6 ) may be viewed in circuit terms as the cascade connec tion of η stages; the yth stage only contains copies of the op erator performing the transformation Mj. Appropriate per mutations will relate the outputs of stage j to the inputs of stage (/ + 1). The use of the permutation χ allows one to vary at will the order of the stages. T h a t flexibility results of the relative independence of the factors in a Kronecker product. Indeed, the operators corresponding to the factor Λ/,· in ( 4 5 ) act on sets of A*/ elements having all their coordinates identical but the /th. The need for interposed shuffles is then justified by the fact that normal factors only act on the least significant coordinate: the shifting property of shuffles is used to bring in turn each of the η coordinates in the least significant position. Proof: Start from the left-hand member of ( 4 6 ) . Replace each Kronecker factor Mi by the equivalent ordinary matrix product of η factors b S k (l^MoKM! For example, if Theorem section, we consider M„-i ® · · · ® Mi ® M 0 the Kronecker (45) a b Mi = c d ef and Mo = α β y IJ. (48) DAVIO: KRONECKER PRODUCTS 123 then \b M\ ® Mo : "α β δ € Ύ! Il TABLE Ι FACTORIZATIONS O F M ® M\ ® M 2 X IA 2 0 7 2 2 fj 7| f1 \a β 17 ι / 0 1 (Λ/ « 2 Λ ® ··® \ I (· · · ® \ Γ 1 Γ 0 ) ( 1 , ® Λ/, 0 2 0 1 2 ® M\ ® 0 1 Λ/ΐ® 2 1 0 2 ( W ®M )(M ® d r ® 2 0 )(1, 2 Γ 0 ® Λ/, β 0 2 0 ICQ) ® 2 2 -S .2 ( 1 β Α/ο) 2 2 = (U*M )-S -(UG 0 Λ/ ) 1, )(Λ/ ® = S2.3- ( 1 ® Μ ι ) 0 ί 2 2 2 Π ® Λ/1 ® 1/·κο)0<·2 lr )(lr R, ® Λ / ) ( Λ / 2 0 2 Μι ® Uo) C2 L R , r o ) ( L R R , <δ lro)(A/ ® 8>Μο)(1 0 Γ ΙΓ,,-ΟΜΙ^Ι ® M )(\ ® 2 0 )Λ/,) 3t3 Our present purpose is however not to give explicit de scriptions of all the possible circuits, but merely to concentrate on some particular cases and to relate these circuits to previ ously described ones. From now on, we restrict ourselves to the particular case where all the M/'s are square matrices of order bj and where χ is the identity mapping on JO, 1, · , η — lj. Under these assumptions, formula (46) may be restated as ® M,-= (1* -,-Α/ ι^/βΐ6Η-Αο). ft i=n— Λ (49) + 1 To shorten the notations, we also define ® · · · ® Mi ® Ck 1 2 /'=/?— 1 ® · · ·) n « 2 The corresponding circuit diagrams are shown in Fig. 6(a) and k ) This rule may be called a quasi-commutative rule: it indeed reduces to the usual commutativity if all the matrices Μ,· are square matrices (of any order). 2) Theorem 9 generalizes to an arbitrary Kronecker product a factorization of the nth Kronecker power of a square matrix used by Lechner [6], [10]. B. Circuit ( Λ / Mi ® M ( ··® M k 0 0 ® · · · ® Mi ® · ·) ®M 1 1 y b) The six factorizations corresponding to η — 3 are given in Table I. Remarks: 1) The ΑΪ! factorizations described by Theorem 8 are actually distinct, since they may in particular involve matrices of different sizes. They may, however, be derived from each other by applying step by step a simple rule, illustrated here for i <k. r k 0 Mi ® M * ® \ M ® M\ ® Mo 1 1 yV α β 0 (50) W e now have Theorem 9: Mi= Application i—n— It now only remains to make a joint use of the results ob tained in Sections II-C and III-A to complete the description of a variety of circuits performing a product Mv when Μ has a Kronecker product structure. It is immediate to observe that all the factors appearing in the right-hand side of (46) are of the type studied in Theorem 7-2) and 3). Accordingly, in the most general situation, we shall be able to exhibit roughly 2 n\ distinct circuits performing the product Mv when Μ is of the form (45): indeed, in each of the n\ factorizations (46) we may replace each of the η factors by either one of its ex pressions (43) or (44). Let us first illustrate our design philosophy by a simple ex ample. Consider the product Μ\ ® M , where M\ is a (2, 3)-matrix and Mo is a (3, 2)-matrix. By (48), we obtain 1 Π [S , (\ ®Mi)]. Bl bi (51) Bi i=n—\ Proof: By replacing in (49) each of the right-hand side factors by its expression (43), one first obtains ο ® ο Π M/= i=n—\ [Sbi-\---b .b„-i---bi 0 i=n—\ • (\ Bi ® Mi)S _ bn _ x bubi ]. x bo n 0 Λ/ι ® M 0 = (Μ, ® 1 ) 2 (1 ®M ) 2 0 In the latter expression, the factors ( 1 ^ . ® M/) and (1^,_, ® M / _ i ) are separated by the product of shuffles Sb -\ n • • • bi,bj-\ - bo' Sb,- • • • bo,b -\ • • • b,-\2 An immediate computation making use of (38) finally shows that the latter product of shuffles is actually equal to ^,-,Λ-i. Q.E.D. The philosophy underlying Theorem 9 is quite simple. Given the initial representation [i -u ' ' ' , i i> *'o] of an input point in the basis [/3 -ι, · · · , b\, bo], one replaces the action of the Kronecker product appearing in the left-hand member of (51) by the consecutive actions of a series of normal factors (1 ^ ® M/), each of which acts on blocks of points differing only by n = ( 1 ® M ) ( M ! ® 1 ). 3 0 3 Let us now apply (42) to M\ ® 1 and to M i ® 1 , respectively. We obtain 2 3 n Λ 124 IEEE T R A N S A C T I O N S O N C O M P U T E R S , VOL. C-30, N O . 2, FEBRUARY MQ Mi M2 MQ Mi M M Mi Mz MQ Mi M Mo M, Mj Mo Mi M Mo Mi Mj Mo Mi M 0 Fig. 6. (b) Circuit diagram for Μ\ ® MQ. (a) (Ι2 ® MQ); 5*2,2; (I2 ® 5 . 3 . (b) 5 , ; ( I 3 ® Μi); S , ; ( 1 ® Λ/ο). 2 2 3 3 3 Fig. 7. MM b · (U»-i ® M)] will easily be recognized as Good's factorization [2]. An equivalent circuit is obtained if one uses (44) instead of (43) to obtain a circuit description of the Kronecker product operation. We shall only illustrate the highlights of the cor responding interpretation in the case where all the matrices Mi have the same order b. In this case, formula (49) may be restated as i—n— Mi= ft 1 i=n— ( 1 { Γ ' - ® Μ , · . ® 1 J'"!). 1 ] (53) 1 If one replaces in (53) each factor of the right-hand side by its expression (44), one obtains ® i=n—\ M,= ft [ U l T ' - ® ^ ) 1 1 Xd^-.^M/Kli"-'- !®^)]. 1 Consider in the latter expression the factors inserted between (U/i-i ® Mj+\) and (\ n-\ ® Mi). These factors are b (U' - / 2 1 0S^ i)(U' " , / Circuit diagrams for Mi ® M\ ® MQ. n 2 M ®Μι®M I 1 + The latter product may be written as ®S6'.6). 0 = 515.2(1 is ® Α/ )5Ίο.3 0 2 X (l,o® Mi)S , (l ® 6 5 6 M ). 0 2) For η = 2 (b = 2), one obtains, similarly, 3 M 2 ® Mi ® M 0 = .S4.2O4 0 ^2)^4,2 X ( 1 ® A / | ) S 4 . 2 ( l 4 ® Af ). 4 0 This circuit is illustrated by Fig. 7(a). The corresponding handling by (53) and (54) yields M 2 ® Mi ® M 0 = £4,2(14 ® ^2)^2,4 X ( h ® S22XI4 ® MDO2 ® S 2 2 X I 4 ® Λ/ο). This circuit is illustrated by Fig. 7(b). IV. i—n— 1 , 2 The latter expression is interpreted as (37). In this approach, the consecutive digits i - 1 , · · , i\, io of the representation of a domain point i are lead in the unit position by a sequence of transpositions of the type (/*, io). The interconnection pattern obviously changes from stage to stage and will easily be rec ognized to belong to the familiar Cooley-Tukey type [3]. Examples: 1) For b = 2, b\ = 3, b = 5, formula (51) yields 2 ® 2 2 ] (52) n th 2 il""'- ®[V'-(U®%)]. r = [S n-i 2 i); 3 the ith digit of their initial representations. These digits are in turn lead to the unit position by consecutive cyclic shifts caused by the shuffles Ss b Observe, furthermore, that if all the matrices Μ,· are of order b, the interstage connection pattern now described by Sb"-\b is constant throughout the circuit. In particular, if all the matrices Μ,· are identical, formula (51) written as if 1981 CONCLUSIONS The essential result obtained in this paper is probably the discovery of a simple algebra able to describe a wide class of switching circuits, ranging from discrete transform circuits to permutation networks. The most interesting property of the described algebra is probably the ease with which one may derive families of equivalent circuits. It is, for example, in teresting to know that the constant geometry patterns, dis- DAVIO: K R O N E C K E R PRODUCTS covered by Good in the context of H a d a m a r d transform, are also applicable to permutation networks in the W a k s m a n style. From a theoretical point of view, we tried to use as few tools as possible and managed to describe the main families of practical circuits using only Kronecker products and perfect shuffles. T h e complexity of (37) and (54), which basically describe an elementary concept (transposition), suggests however, an enrichment of our set of tools. This would probably provide us with more concise and more sensible circuit de scriptions. ACKNOWLEDGMENT T h e author is indebted to Miss C. Gossart for fruitful dis cussions. H e also wishes to thank the referees of the paper for extremely detailed and fruitful comments and criticisms. REFERENCES [1] C. K . Rushford, "Fast Fourier Hadamard decoding of orthogonal codes," Inform. Contr., vol. 15, pp. 33-47, 1969. [2] I. J. Good, "The interaction algorithm and practical Fourier analysis," J. Roy. Stat. Soc, ser. B, vol. 20, pp. 361-372,1958. [3] J. W. Cooley and J. W. Tukey, "An algorithm for the machine com putation of complex Fourier," Series. Math. Comput., vol. 19, pp. 297-301, 1965. [4] M. C. Pease, "An adaptation of the fast Fourier transform for parallel processing," J. Ass. Comput. Mach., vol. 15, pp. 252-264, 1968. [5] I. J. Good, "The relationship between two fast Fourier transforms," IEEE Trans. Comput., vol. C-20, pp. 310-317, 1971. [6] R. J. Lechner, "Harmonic analysis of switching functions," in Recent Developments in Switching Theory, A. Mukhopadhyay, Ed. New York: Academic, 1971. [7] A. Waksman, "A permutation network," J. Ass. Comput. Mach., vol. 15, pp. 159-163, 1968. [8] H. S. Stone, "Parallel processing with the perfect shuffle," IEEE Trans. Comput., vol. C-20, pp. 153-161, 1971. [9] J. Lenfant, "Parallel permutations of data: A Benes network control algorithm for frequently used permutations," IEEE Trans. Comput., vol.C-27, pp. 637-647, 1978. [10] R.J. Lechner, "Transformations among switching function canonical forms," IEEE Trans. Electron. Comput., vol. EC-12, no. 2, pp. 129-130, 1963. 125 [11] M. Davio, J. P. Deschamps, and A. Thayse, Discrete and Switching Functions. New York: McGraw-Hill, 1978. [12] B. J. Fino and R. Algazi, "A unified treatment of discrete fast unitary transforms," SI AM J. Comput., vol. 6, pp. 700-717, 1977. [13] E. O. Brigham, The Fast Fourier Transform. Englewood Cliffs, NJ: Prentice-Hall, 1974. [14] F. Yates, The Design and Analysis of Factorial Experiments. Harpenden: Imperial Bureau of Soil Science, 1937. [15] D. E. Knuth, The Art of Computer Programming. Vol. 2: Seminumerical Algorithms. Reading, MA: Addison-Wesley, 1969. [16] M. Davio and J. P. Deschamps, "Addition in signed digit number sys tems," in Proc. 8th Int. Conf. on Multiple Valued Logics, Chicago, IL, 1978, pp. 104-113. [17] Ν. S. Szabo and R. J. Tanaka, Residue Arithmetic and its Applications to Computer Technology. New York: McGraw-Hill, 1967. [18] R. Bellman, Introduction to Matrix Analysis. New York: McGrawHill, 1960. [19] S. W. Golomb, "Permutations by cutting and shuffling," SI AM Rev., vol.3, pp. 293-297, 1961. [20] J. B. Kam and G. I. Davida, "Structured design of substitution-per mutation encryption networks," IEEE Trans. Comput., vol. C-28, pp. 747-753, Oct. 1979. [21] S. B. Morris and R. E. Hartwig, "The generalized Faro shuffle," Discrete Math., vol. 15, pp. 333-346, 1976. [22] I. Cahit, "Realization of graceful permutations by a shuffle-exchange network," Informat. Process. Lett., vol. 6, pp. 171-173, 1977. Marc Davio was born in Monceau-sur-Sambre, Belgium, in June 1938. He received the M.Sc. and Ph.D. degrees in electrical engineering from the University of Louvain, Louvain, Belgium, in 1961 and 1968, respectively. He is currently a Research Group Leader at the Philips Research Laboratory, Brussels, and Pro fessor at the University of Louvain, Louvain-laNeuve. He is engaged in research on digital circuit design and algorithm implementation.