* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download cg-type algorithms to solve symmetric matrix equations
Survey
Document related concepts
Eigenvalues and eigenvectors wikipedia , lookup
Jordan normal form wikipedia , lookup
Determinant wikipedia , lookup
Four-vector wikipedia , lookup
Singular-value decomposition wikipedia , lookup
Matrix (mathematics) wikipedia , lookup
Perron–Frobenius theorem wikipedia , lookup
Orthogonal matrix wikipedia , lookup
System of linear equations wikipedia , lookup
Matrix calculus wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Cayley–Hamilton theorem wikipedia , lookup
Transcript
CG-TYPE ALGORITHMS TO SOLVE SYMMETRIC MATRIX EQUATIONS Davod Khojasteh Salkuyeh Department of Mathematics, Mohaghegh Ardabili University, P. O. Box. 56199-11367, Ardabil, Iran E-mail: [email protected] Abstract The global FOM and GMRES are among the effective algorithms to solve linear system of equations with multiple right-hand sides. In this paper, we study these algorithms in the case that the coefficient matrix is symmetric and extract two CGtype algorithms for solving symmetric linear systems of equations with multiple right– hand sides. Then, we compare the numerical performance of the new algorithms with some available methods. Numerical experiments are done on some test matrices from Harwell-Boeing collection. AMS Subject Classification : 65F10. Keywords: Global FOM and GMRES algorithms; global CG and CR algorithms; matrix equations; Krylov subspace. 1. Introduction The Conjugate Gradient (CG) algorithm [4] is an extremely effective method to solve symmetric positive definite (SPD) linear system of equations. This algorithm can be extracted from the full orthogonalization method (FOM) [7] in the case that the coefficient matrix is an SPD matrix. Also, the GMRES algorithm [7, 8] results in the conjugate residual (CR) algorithm [7] to solve symmetric linear system of equations not necessarily positive definite. It is well known that the CG and CR algorithms are less cost effective than FOM and GMRES algorithms, respectively. Consider the linear system of equations AX = B, 1 (1) where A is a symmetric matrix of order n and B = [b(1) , · · · , b(s) ] is an arbitrary rectangular matrix of order n×s with s of moderate size (s n) and X = [x(1) , · · · , x(s) ]. There are alternative strategies for solving (1) by iterative methods. In [6], the block CG (Bl-CG) method has been presented when A is an SPD matrix. Another method which is based on Krylov subspace methods has been proposed in [9] for linear system of equations with general coefficient matrices. Recently, K. Jbilou et al. have proposed the global FOM (Gl-FOM) and global GMRES (Gl-GMRES) algorithms which are based on block Krylov subspace methods [5]. These methods can be applied for solving Lyapunov and Sylvester matrix equations [1, 3, 5] as well. In this paper, two CG-type algorithms are extracted to solve (1) from Gl-FOM and Gl-GMRES in the case that the matrix A is SPD and symmetric, respectively. Throughout this paper, we use the following notations. Let E = Rn×s . For X and Y in E, we define the inner product (X, Y )F = tr(X T Y ), where tr(Z) denotes the trace of the square matrix Z and X T denotes the transpose of the matrix X. The associated norm is the well-known Frobenius norm denoted by k . kF . For a matrix V ∈ E the matrix Krylov subspace Km (A, V ) is defined as follows Km (A, V ) = span{V, AV, · · · , Am−1 V }, which is a subspace of E. A set of members of E is said to be F -orthogonal if it is orthogonal with respect to the scalar product (., .)F . This paper is organized as follows. In section 2, we give a brief description of Gl-FOM and Gl-GMRES algorithms. The new CG-type algorithms are presented in section 3. Section 4 is devoted to numerical experiments. Some conclusion remarks are given in section 5. 2. A brief description of Gl-FOM and Gl-GMRES algorithms Let V ∈ E, we note that Z ∈ Km (A, V ) means that Z= m−1 X αk Ak V. k=0 The Modified global Arnoldi algorithm constructs an F -orthogonal basis V1 , V2 , · · · , Vm , i.e., tr(ViT Vj ) = 0 for i 6= j, i, j = 1, · · · , m, and tr(ViT Vi ) = 1, of the Krylov subspace Km (A, V ). The algorithm can be written as following. 2 Algorithm 1. Modified global Arnoldi algorithm 1. Choose an n × s matrix V1 such that k V1 kF = 1. 2. For j = 1, 2, ..., m Do: 3. Compute W := AVj 4. For i = 1, ..., j Do: 5. hij := tr(ViT W ), 6. W := W − hi,j Vi 7. EndDo 8. hj+1,j :=k W kF . 9. Vj+1 := W/hj+1,j 10. EndDo From this algorithm we have an upper Hessenberg matrix of order (m + 1) × m, e Hm , such that its entries are h(i, j) = trace(ViT AVj ) = (AVj , Vi )F , i = 1, · · · , m, j = 1 · · · , m + 1. As in [5], we define some notations. By Vm we define the n × ms matrix Vm = e m denotes the (m + 1) × m upper Hessenberg matrix whose [V1 , V2 , · · · , Vm ]. Also, H nonzero entries hi,j are defined by Algorithm 1 and Hm is the m × m matrix obtained e m by deleting its last row. H.,j will denote the jth column of the matrix Hm . from H Next, we use the notation ∗ for the following product Vm ∗ α = m X αk Vk , (2) k=0 where α = (α1 , α2 , · · · , αm )T is a vector in Rm and for a matrix Hm of dimension m × m we define Vm ∗ Hm = [Vm ∗ H.,1 , Vm ∗ H.,2 , · · · , Vm ∗ H.,m ]. (3) It can be easily seen that Vm ∗ (α + β) = Vm ∗ α + Vm ∗ β and (Vm ∗ Hm ) ∗ α = Vm ∗ (Hm α) (4) where α and β are two vectors in Rm . Using these notations, we have the following results which will be used later. Proposition 1. Let Vm be the matrix defined by Vm = [V1 , V2 , . . . , Vm ] where the 3 n × s matrices Vi , i = 1, . . . , m, are defined by the global Arnoldi algorithm. Then we have (5) k Vm ∗ α kF =k α k2 where α is a vector of Rm . Proof. See [5]. e m as defined before, then using the product ∗, the Theorem 1. Let Vm , Hm and H following relations hold: (6) AVm = Vm ∗ Hm + Zm+1 , where Zm+1 = hm+1,m [0n×s , . . . , 0n×s , Vm+1 ], and em. AVm = Vm+1 ∗ H (7) Proof. See [5]. In [5], K. Jbilou et al. proposed the global FOM and GMRES algorithms which are based on the modified global arnoldi Algorithm, for solving linear systems with multiple right-hand sides. In this section we review the Gl-FOM and Gl-GMRES algorithms. Let X0 be an initial n × s matrix guess to the solution X of the equation (1) and R0 = B − AX0 its associated residual. At the mth iterate of Gl-FOM algorithm, a correction Zm is determined in the matrix Krylov subspace Km (A, R0 ) = Km such that the new residual is F -orthogonal to Km , i.e., Xm − X0 = Zm ∈ Km (A, R0 ) (8) Rm = R0 − AZm ⊥F Km (A, R0 ). (9) The equation (8) yields Zm = Vm ∗ym , where ym is a vector in Rm . Hence the residual Rm is given by Rm = R0 − AVm ∗ ym , (10) where R0 = B − AX0 . Suppose β =k R0 kF and V1 = V1 /β. Now, since Rm is F -orthogonal to Km , we get (11) Hm ym = βe1 where e1 is the first canonical basis vector in Rm . As the FOM algorithm, these relations result in the Gl-FOM algorithm. This algorithm can be written as following. 4 Algorithm 2. Gl-FOM algorithm 1. Choose X0 and compute R0 = B − AX0 , β :=k R0 k2 and V1 := R0 /β. 2. For j = 1, 2, ..., m for ym and construct the matrices V1 , V2 , ... by Algorithm 1 and set Vm = [V1 , V2 , · · · , Vm ]. 3. Solve Hm ym = βe1 for ym and compute Xm = X0 + Vm ∗ ym . It can be easily seen that Rm = B − AXm = −hm+1,m eTm ym Vm+1 . (12) Any approximate solution Xm of the problem (1) in X0 + Km can be written as Xm = X0 + Vm ∗ ym , (13) where ym ∈ Rm . From the Eq. (10) and if V1 = R0 /β then we have k Rm kF = k B − AXm kF = k R0 − AVm ∗ ym kF e m ym ) kF = k Vm+1 ∗ (βe1 − H e m ym k2 . = k βe1 − H from Theorem 1 from Proposition 1 e k y k2 , i.e., Now, vector ym is chosen the minimizer of k βe1 − H e k y k2 . ym = argminy∈Rm k βe1 − H (14) By using the above discussions the Gl-GMRES algorithm can be summarized as following. Algorithm 3. Gl-GMRES algorithm 1. Choose X0 and compute R0 = B − AX0 , β :=k R0 kF and V1 := R0 /β. 2. For j = 1, 2, ..., m for ym and construct the matrices V1 , V2 , ... by Algorithm 1 and set Vm = [V1 , V2 , · · · , Vm ]. 3. Compute ym the minimizer of k βe1 −H m y k2 for y ∈ Rm and Xm = X0 +Vm ∗ym . 3. Global CG and global CR algorithms 5 In this section we study the Gl-FOM and Gl-GMRES in the case that the coefficient matrix A is symmetric. First, we state the following theorem. Theorem 2. Assume that global Arnoldi’s method is applied to a real symmetric matrix A. Then the matrix Hm generated by the algorithm is tridiagonal and symmetric. Proof. The matrix Hm is a Hessenberg matrix. On the other hand we have hij = tr(ViT AVj ) = tr(VjT AVi ) = hji . Hence, the matrix Hm is tridiagonal and symmetric. This theorem shows that the matrix Hm can be written as following α1 β2 β2 α2 β3 . . . . Hm = βm−1 αm−1 βm βm αm This leads the following form of the global modified Arnoldi’s method, namely the global Lanczos algorithm. Algorithm 4. Global Lanczos algorithm 1. Choose an n × s matrix V1 such that k V1 kF = 1. Set β1 = 0, V0 = 0. 2. For j = 1, 2, ..., m do 3. Wj := AVj − βj Vj−1 (If j = 1 set β1 V0 ≡ 0) 4. αj := tr(VjT Wi ) 5. Wj := Wj − αj Vj 6. βj+1 :=k Wj kF . If βj+1 = 0 then stop. 7. Vj+1 := Wj /βj+1 . 8. Enddo By using the global Lanczos and FOM algorithms we can derive a new algorithm for solving a symmetric linear systems with multiple right-hand sides. Algorithm 5. 1. Compute R0 = B − AX0 , β :=k R0 kF and V1 := R0 /β. 2. At step m use the Laczos algorithm to get an F -orthogonal basis of Km . 6 3. Set Hm = tridiag(βi , αi , βi+1 ) and Vm = [V1 , . . . , Vm ]. −1 4. Compute ym = Hm (βe1 ) and Xm = X0 + Vm ∗ ym Now, let A be an SPD matrix. In this case the LU factorization of Hm exists and can be written as Hm = Lm Um where η1 β2 1 λ2 1 η2 β3 . . 1 λ . . 3 (15) Hm = . . . . . . . . . ηm−1 βm λm 1 ηm Hence, the approximate solution Xm is given by −1 −1 Xm = X0 + Vm ∗ (Um Lm βe1 ) −1 = X0 + (Vm ∗ Um ) ∗ (L−1 m βe1 ). from relation (4) Letting −1 Pm = Vm ∗ Um , (16) zm = L−1 m βe1 , (17) Xm = X0 + Pm ∗ zm , (18) and then where Pm = [P1 , . . . , Pm ]. It can be easily seen that Pm ∗ Um = Vm , (19) [Pm ∗ (Um ).,1 , . . . , Pm ∗ (Um ).,m ] = Vm . (20) or So the latter relation follows that Pm ∗ (Um ).,m = Vm , (21) βm Pm−1 + ηm Pm = Vm , (22) and it follows that or Pm = 1 [Vm − βm Pm−1 ]. ηm 7 (23) On the other hand from Eq. (14) we have λm = βm , ηm−1 (24) ηm = αm − λm βm . Now, let zm = zm−1 ζm , (25) (26) then we have ζm = −λm ζm−1 . Therefore we can conclude that Xm = Xm−1 + ζm Pm , (27) where Pm is defined by Eq. (23). This gives the global D-Lanczos algorithm which refers to the direct version of global Lanczos algorithm for solving Eq. (1). Algorithm 6. Global D-Lanczos algorithm 1. Compute R0 = B − AX0 , ζ1 := β :=k R0 kF , and V1 := R0 /β 2. Set λ1 = β1 = 0, P0 = 0 3. For m = 1, 2, . . . , until convergence do: 4. Compute W = AVm − βm Vm−1 and αm := (W, Vm )F 5. If m > 1 then compute λm = 6. ηm = αm − λm βm 7. −1 Pm = ηm (Vm − βm Pm−1 ) 8. Xm = Xm−1 + ζm Pm 9. If xm has converged then Stop βm ηm−1 10. W := W − αm Vm 11. βm+1 :=k W kF , Vm+1 = W/βm+1 and ζm = −λm ζm−1 12. Enddo To derive the Gl-CG algorithm from the Global D-Lanczos algorithm we state and prove the following proposition. Proposition 2. Let Rm = B − AXm , m = 0, 1, . . . , be the residual matrices and Pm , m = 0, 1, . . . , be the auxiliary matrices produced by the global D-Lanczos algorithm. Then, 8 1. Each residual matrix Rm is such that Rm = σm Vm+1 where σm is a certain scalar. As a result the Residual matrices are F -orthogonal to each other. 2. The auxiliary matrices Pm are A-conjugate set with respect to inner product (., .)F , i.e. (APi , Pj )F = 0 for i 6= j. Proof. It is obvious that the two algorithms 5 and 6 are mathematically equivalent. Hence the first part of the proposition immediate consequence of the relation (12). −1 in (16), then For the secondP part, let [δ1j , . . . , δjj , 0, . . . , 0]T be the jth column of Um j we have Pj = k=1 δkj Vk . Therefore, (APi , Pj )F = (A = i X δli Vl , l=1 j i XX j X δkj Vk )F k=1 δlj δkj (AVl , Vk )F . l=1 k=1 Since (AVl , Vk )F = 0, for l 6= k we conclude that (APi , Pj )F = 0, for i 6= j. By using this proposition we can obtain the Gl-CG algorithm. From the global D-Lanczos algorithm we have Xj+1 = Xj + αj Pj . (28) Therefore the residual matrix must satisfy the recurrence Rj+1 = Rj − αj APj . (29) From the F -orthogonality of Rj ’s we get (Rj − αj APj , Rj )F = 0 and as a result αj = (Rj , Rj )F . (APj , Rj )F (30) Also, it is known that the next search direction Pj+1 is a linear combination of Rj+1 and Pj (step 7 of the global Lanczos algorithm), i.e., Pj+1 = Rj+1 − βj Pj . (31) Thus, a consequence of the above relation is that (APj , Rj )F = (APj , Pj − βj−1 Pj−1 )F = (APj , Pj )F , 9 (32) since APj is F -orthogonal to Pj−1 . Hence the relation (30) becomes (Rj , Rj )F . (APj , Pj )F αj = (33) From the F -orthogonality of Pj+1 to APj yields βj = − (Rj+1 , APj )F . (Pj , APj )F (34) 1 (Rj+1 − Rj ), αj (35) From the equation (29) we get APj = − and hence 1 (Rj+1 , Rj+1 − Rj )F (Rj+1 , Rj+1 )F = . αj (APj , Pj )F (Rj , Rj )F The above relations result in the Gl-CG algorithm. βj = − (36) Algorithm 7. Global CG algorithm 1. Compute R0 = B − AX0 , P0 := R0 . 2. For j := 0, 1, . . . , until convergence do 3. αj := (Rj , Rj )F /(APj , Pj )F 4. Xj+1 := Xj + αj Pj 5. Rj+1 := Rj − αj APj 6. βj := (Rj+1 , Rj+1 )F /(Rj , Rj )F 7. Pj+1 := Rj+1 + βj Pj 8. Enddo As we see the CG algorithm is an special case of Gl-CG Algorithm for SPD linear system of equations with a single right-hand side. A new algorithm can be derived from the Gl-GMRES algorithm when the coefficient matrix is symmetric (like Gl-CG from Gl-FOM). In this case, we have (APi , APj )F = 0, for i 6= j, (ARi , Rj )F = 0, for i 6= j. and If we look for an algorithm with same structure as Gl-CG, but with these conditions we get the Global Conjugate Residual (Gl-CR) algorithm. 10 Algorithm 8. Global CR algorithm 1. Compute R0 = B − AX0 , P0 := R0 . 2. For j := 0, 1, . . . , until convergence do 3. αj := (Rj , ARj )F /(APj , APj )F 4. Xj+1 := Xj + αj Pj 5. Rj+1 := Rj − αj APj 6. βj := (Rj+1 , ARj+1 )F /(Rj , ARj )F 7. Pj+1 := Rj+1 + βj Pj 8. APj+1 := ARj+1 + βj APj 9. Enddo 4. Numerical examples All the numerical experiments presented in this section were computed in double precision with some FORTRAN 77 codes. For all the examples the initial guess X0 was taken to be the zero matrix. The right hand side B was chosen such that the exact solution X is a matrix of order n × s whose ith column has all entries equal to one except the ith entry which is zero. The tests were stopped as soon as the stopping criterion k b(i) − Ax(i) k2 < 10−7 , max 1≤i≤n k B kF is satisfied. For the first set of numerical experiments, some SPD matrices from HarwellBoeing collection [2] were used. These matrices with their generic properties are given in Table 1. In this table, n and nnz stand for the order and the number of nonzero entries of A, respectively. Table 2 also reports the numerical performance of the Gl-CG algorithm and CG algorithm for these matrices applied to a sequence of right-hand sides. In this table, we listed the number of iterations to converge for s = 2, 4, 8, 16 and 32. We also give the CPU times to converge in parentheses. For the second set of our numerical experiments some unsymmetric matrices from Harwell-Boeing collection are chosen and their symmetrized part are considered, i.e., e were shown. In this table nnz e := 1 (A + AT ). In Table 2, some properties of A A 2 stands for the number of nonzero entries of symmetrized matrices. Table 2 also 11 presents the numerical results of the Gl-CR algorithm and CR algorithm for these matrices applied to a sequence of right-hand sides. Table 1 : Numerical results for SPD matrices n nnz method s=2 s=4 s=8 s = 16 s = 32 NOS5 matrix 468 5172 NOS6 675 3255 NOS7 729 4617 GR-30-30 900 7744 PDE900 900 7860 BCSSTK06 420 7860 BCSSTK08 1074 12960 BCSSTK09 1083 18437 BCSSTK10 1086 22070 BCSSTK11 1473 34241 BCSSTK19 817 6853 Gl-CG CG Gl-CG CG Gl-CG CG Gl-CG CG Gl-CG CG Gl-CG CG Gl-CG CG Gl-CG CG Gl-CG CG Gl-CG CG Gl-CG CG 336(0.50) 368(0.77) 520(0.93) 572(1.32) 4009(5.44) 4347(7.63) 52(0.17) 52(0.22) 1316(2.86) 1251(3.24) 2397(3.35) 2601(4.84) 2392(6.48) 2595(9.23) 199(1.15) 198(1.32) 2356( 9.89) 2361(13.51) 2573(18.40) 2724(21.86) 1368(2.69) 1979(4.39) 365(1.37) 366(1.43) 494(1.54) 590(2.03) 4027(10.66) 4560(14.61) 52(0.44) 53(0.49) 1241(5.21) 1247(6.10) 1766(4.84) 2719(8.74) 2403(14.22) 2628(18.35) 199(1.92) 199(2.15) 2270(19.28) 2304(25.27) 2429(35.54) 2583(41.47) 979(3.79) 2097(8.73) 351(2.15) 357(2.41) 450(2.85) 547(3.79) 4023(26.75) 4744(34.38) 52(0.82) 52(0.94) 1241(12.19) 1306(13.46) 1686( 9.01) 2655(17.19) 2136(28.01) 2567(37.73) 218(3.90) 220(4.34) 2155(42.78) 2254(49.55) 2330(73.16) 2380(77.23) 852(7.63) 2560(24.39) 346(4.4) 345(4.5) 470(6.53) 521(7.41) 2784(42.34) 3780(59.82) 50(1.54) 51(1.65) 1226(25.81) 1303(27.85) 1713(21.04) 2699(36.14) 2068(58.77) 2775(84.70) 215(7.47) 214(8.35) 2099(82.06) 2094(96.40) 2299(144.13) 2304(154.34) 796(15.70) 2561(50.53) 328(8.46) 344(9.11) 454(13.29) 521(15.55) 2807( 92.06) 3780(127.92) 49(2.64) 49(2.80) 1221(53.61) 1300(58.44) 1692(44.21) 2601(75.08) 1815(104.25) 2744(172.30) 201(13.85) 210(15.76) 2025(164.78) 2061(190.54) 2170(273.25) 2300(310.77) 603(24.94) 2120(87.71) Table 2 : Numerical results for symmetric matrices n nnz method s=2 s=4 s=8 s = 16 s = 32 CAVITY01 matrix 317 6589 CAVITY02 317 3851 CAVITY03 317 6627 CAVITY04 317 3851 CAVITY05 1182 26004 CAVITY06 1182 20026 SHERMAN1 1000 3750 SHERMAN4 1104 3786 GRE185 185 1425 GRE343 343 2277 west0132 132 686 Gl-CR CR Gl-CR CR Gl-CR CR Gl-CR CR Gl-CR CR Gl-CR CR Gl-CR CR Gl-CR CR Gl-CR CR Gl-CR CR Gl-CR CR 486(1.37) 483(1.37) 87(0.11) 87(0.17) 3832(7.36) 4993(9.45) 263(0.55) 277(0.60) 970(9.67) 980(9.88) 160(1.59) 160(1.65) 389(1.32) 388(1.43) 127(0.49) 127(0.55) 204(0.22) 207(0.22) 531(0.94) 540(0.99) 2192(1.15) 2663(1.43) 486(2.20) 490(2.20) 87(0.33) 87(0.39) 4849(18.18) 4716(17.85) 260(1.15) 276(1.21) 799(16.09) 785(16.20) 159(2.80) 159(2.91) 377(2.36) 376(2.47) 126(1.27) 126(1.27) 209(0.33) 208(0.38) 533(1.53) 528(1.54) 2051(1.81) 2290(1.97) 494(4.32) 489(4.40) 86(0.77) 90(0.98) 3492(26.92) 5360(42.02) 277(2.03) 274(1.94) 710(28.35) 710(29.22) 157(5.39) 150(5.44) 366(4.95) 364(5.22) 124(2.14) 124(2.25) 208(0.71) 216(0.99) 553(2.69) 530(2.75) 1788(2.80) 2105(3.35) 485(8.57) 496(8.85) 86(1.49) 89(1.59) 2425(40.97) 3820(65.86) 266(3.62) 273(3.85) 580(48.71) 597(49.16) 156(10.44) 156(10.82) 357( 9.67) 354(10.27) 123(3.90) 123(4.28) 208(1.48) 215(1.54) 567(5.88) 524(5.76) 1506(4.45) 1951(5.71) 431(15.70) 446(16.36) 85(2.63) 87(2.81) 2314( 83.05) 3327(120.18) 272(7.47) 272(7.64) 537(86.18) 536(88.21) 155(20.43) 155(22.24) 347(19.50) 343(20.32) 122(7.74) 122(8.02) 210(2.91) 212(3.02) 556(12.8) 540(12.52) 1295(8.79) 1828(12.68) 12 Table 1 and 2 shows that the results of Gl-CG and Gl-CR algorithms is often better than that the CG and CR algorithms applied to the sequence of right-hand sides, both CPU times and iteration. Numerical examples show that the Gl-CG is less cost effective than the Bl-CG algorithm for solving SPD linear systems. To validate our claim we display the convergence history in Figures 1 and 2 for the systems corresponding to the matrices NOS5 and BCSSTK09, respectively, in the first set of our numerical examples. 5. Conclusion In this paper, we have extracted the global CG and CR algorithms from the global FOM and GMRES for solving matrix equations, respectively. Our numerical examples show that the global CG and CR are often less cost effective than that of the CG and CR algorithms applied to a sequence of right-hand sides. Numerical results also show that the Bl-CG algorithm is more cost effective than global CG algorithm for solving SPD linear matrix equations. We also note that the global CG and CR algorithms can be directly applied to solve the symmetric Sylvester matrix equation but the CG or CR algorithms can not. References [1] R. H. Bartels, G. W. Stewart, Algorithm 432: Solution of the matrix equation AX+XB=C, Circ. Syst. and Signal Proc., 13(1994)820-826. [2] I. S. Duff, R. G. Grimes, J. G. Lewis, Users’ Guide for the Hawell-Boeing sparse matrix collection,, Technical Report RAL-92-086, Rutherford Applton Laboratory, Chilton, UK, 1992. [3] G. H. Golub, S. Nash, C. Van Loan, A Hessenberg-Schur method for the problem AX+XB=C, IEEE Trans. Autom. Contr., AC-24(1979)909-913. [4] M. R. Hestenes, E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Standards, 49(1952)409-436. [5] K. Jbilou, A. Messaoudi and H. Sadok, Global FOM and GMRES algorithms for matrix equations, Appl. Numer. Math. 31(1999)49-63. 13 8 Gl−CG Bl−CG 7 6 log10(||Rk||F) 5 4 3 2 1 0 0 200 400 600 Iterations 800 1000 1200 Figure 1: Convergence history of Gl-CG and Bl-CG for NOS5 with s = 4 9 8 Gl−CG Bl−CG 7 log10(||Rk||F) 6 5 4 3 2 1 0 200 400 600 Iterations 800 1000 1200 Figure 2: Convergence history of Gl-CG and Bl-CG for BCSSTK09 with s = 4 14 [6] D. O’Leary, The block conjugate algorithm and related methods, Linear Algebra Appl. 29(1980)293-322. [7] Y. Saad, Iterative Methods for Sparse linear Systems (PWS press, New York,1995) [8] Y. Saad and M.H. Schultz, GMRES, A generalized minimal residual algorithm for nonsymmetric linear systems, SIAM J. Sci. Statist. Comput. 7(1986)856-869. [9] V. Simoncini and E. Gallopoulos, An iterative method for nonsymmetric systems with multiple right-hand sides, SIAM J. Sci. Comput. 16(1995)917-993. 15