* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download The Sine Transform Operator in the Banach Space of
Survey
Document related concepts
Rotation matrix wikipedia , lookup
Covariance and contravariance of vectors wikipedia , lookup
System of linear equations wikipedia , lookup
Determinant wikipedia , lookup
Symmetric cone wikipedia , lookup
Eigenvalues and eigenvectors wikipedia , lookup
Matrix (mathematics) wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Singular-value decomposition wikipedia , lookup
Jordan normal form wikipedia , lookup
Orthogonal matrix wikipedia , lookup
Perron–Frobenius theorem wikipedia , lookup
Gaussian elimination wikipedia , lookup
Matrix calculus wikipedia , lookup
Four-vector wikipedia , lookup
Transcript
The Sine Transform Operator in the Banach Space of Symmetric Matrices and Its Application in Image Restoration ∗ Kit-Ian KOU, Xiao-Qing JIN and Vai-Kuong SIN Faculty of Science and Technology University of Macau, Macau Abstract. In this paper, we study an operator s which maps every n-by-n symmetric matrix An to a matrix s(An ) that minimizes kBn − An kF over the set of all matrices Bn that can be diagonalized by the sine transform. The matrix s(An ), called the optimal sine transform preconditioner, is defined for any n-by-n symmetric matrices An . The cost of constructing s(An ) is the same as that of optimal circulant preconditioner c(An ) which is defined in [8]. The s(An ) has been proved in [6] to be a good preconditioner in solving symmetric Toeplitz systems with the preconditioned conjugate gradient (PCG) method. In this paper, we discuss the algebraic and geometric properties of the operator s, and compute its operator norms in Banach spaces of symmetric matrices. Some numerical tests and an application in image restoration are also given. Keywords. Toeplitz matrix, Optimal sine transform preconditioner, Sine transform operator, PCG method, Image restoration AMS(MOS) Subject Classification. 65F10, 65F15, 43E10 ∗ Research supported by the research grant No. RG018/97-98S/JXQ/FST from the University of Macau 1 Sine Transform Operator 1 2 Introduction In this paper, we discuss the optimal sine transform preconditioner which is defined in [6] from the operator point of view. Let (Fn×n , k·k) be the Banach space of all n-by-n symmetric matrices over the real field, equipped with a matrix norm k · k. Let (Bn×n , k · k) be the subspace of all n-by-n symmetric matrices which can be diagonalized by the sine transform, i.e., Bn×n = {Sn Λn Sn | Λn is an n-by-n real diagonal matrix} (1) where Sn is the discrete sine transform matrix. Let s be an operator defined on ( Fn×n , k · k ) such that for any An ∈ Fn×n , s(An ) is the minimizer of kBn −An kF over all Bn ∈ Bn×n . Here k·kF denotes the Frobenius norm. The s is an operator from (Fn×n , k · k) into its subspace (Bn×n , k · k). We call s the sine transform operator and s(An ) the optimal sine transform preconditioner respectively. We remark that the idea can be easily extended to the block matrix Amn by considering the minimizer s(Amn ) of kBmn − Amn kF over all Bmn ∈ Bmn×mn where Bmn×mn = {(Sm ⊗ Sn )Λmn (Sm ⊗ Sn ) | Λmn is diagonal} , (2) see [3] and [14]. Since only the sine transform will be involved, all the computations can be done in real arithmetic. We note that the matrix-vector product Sn v can be done in O(n log n) real operations by using the fast sine transform (FST), see [16]. The matrix s(An ) is equal to An itself if An is a tridiagonal Toeplitz matrix. It is shown by Huckle in [11] that the minimizer s(An ) is given by Sn ΛSn where Λ is the diagonal matrix with diagonal entries Λjj = (Sn An Sn )jj , j = 1, . . . , n. (3) Computing all the diagonal entries of Λ by using (3) will require O(n2 log n) operations. However, R. Chan, M. Ng and C. Wong in [6] show that the minimizer s(An ) can be obtained in O(n2 ) operations for general symmetric matrices and the cost can be reduced to O(n) operations when An is a Toeplitz matrix. These operation counts are the same as that for obtaining optimal circulant preconditioner c(An ) which is defined by T. Chan in [?]. These preconditioners have been proved to be good preconditioners in solving Sine Transform Operator 3 symmetric Toeplitz systems with the PCG method. For the history of the iterative Toeplitz solvers and the applications of the optimal sine transform preconditioners, we refer to [1, 2, 4, 6, 7, 8, 11, 12, 13, 15]. The purpose of this paper is to discuss some other aspects of this operator s. The outline of the paper is as follows. In section 2, we prove some algebraic and geometric properties of the operator s. In section 3, we compute its operator norms in different Banach spaces of symmetric matrices. Some numerical results are given in section 4 and an application in image restoration is given in the last section. 2 The Sine Transform Operator Firstly, let Sn denote the n-by-n discrete sine transform matrix with the (i, j)th entry given by s µ ¶ πij 2 sin , n+1 n+1 1 ≤ i, j ≤ n. We remark that Sn is symmetric and orthogonal, i.e., Sn = Snt and Sn2 = In . Also, for any n-vector v, the matrix-vector multiplication Sn v can be performed in O(n log n) real operations by using the FST, see [16]. Hence the number of operations required for the FST is less than that of the FFT. It was shown in [1] and [12] that a matrix belongs to Bn×n defined by (1) if and only if the matrix can be expressed as a special sum of a Toeplitz matrix and a Hankel matrix. We recall that a matrix An = [ai,j ] is said to be Toeplitz if ai,j = ai−j and Hankel if ai,j = ai+j . For any An in Fn×n , let δ(An ) denote the diagonal matrix whose diagonal is equal to the diagonal of the matrix An . The following Lemma gives a method for finding s(An ), see [11]. Lemma 2.1. Let An = [aij ] ∈ Fn×n and s(An ) be the minimizer of kBn − An kF over all Bn ∈ Bn×n . Then s(An ) is uniquely determined by An and is given by s(An ) = Sn δ(Sn An Sn )Sn (4) Sine Transform Operator 4 where Sn is the sine transform matrix. Hence the eigenvalues of s(An ) are given by δ(Sn An Sn ). Now, we want to discuss some properties of the sine transform operator. The following Theorem is on the algebraic properties of the operator s. Theorem 2.1. We have • (i) For all An , Cn ∈ Fn×n and α, β are real scalars, s(αAn + βCn ) = αs(An ) + βs(Cn ). Moreover, s2 (An ) = s(s(A)) = s(An ) for ∀ An ∈ Fn×n . Thus s is a linear projection operator. • (ii) Let An ∈ Fn×n , then tr(s(An )) = tr(An ) = n−1 X λj (An ) j=0 where λj (An ) are the eigenvalues of An . • (iii) Let An ∈ Fn×n and Bn ∈ Bn×n , then s(Bn An ) = Bn · s(An ) and s(An Bn ) = s(An ) · Bn . • (iv) Let An ∈ Fn×n , then s(A2n ) − [s(An )]2 ≥ 0. Proof: The proofs of (i) and (ii) are trivial, therefore we omit them. The proof of (iii) is as follows. For An ∈ Fn×n and Bn ∈ Bn×n , Bn could be written as Bn = Sn Λn Sn where Λn is a diagonal matrix. By using (4), we have s(Bn An ) = Sn δ(Sn (Bn An )Sn )Sn = Sn δ[Sn (Sn Λn Sn )An Sn ]Sn = Sn δ[Λn (Sn An Sn )]Sn = Sn Λn δ(Sn An Sn )Sn = Sn Λn Sn · Sn δ(Sn An Sn )Sn = Bn · s(An ). Sine Transform Operator 5 By similar arguments, one can prove that s(An Bn ) = s(An ) · Bn . The proof of (iv) is as follows. Let An = [aij ], [Sn ]ij = ξij = sin ³ πij n+1 ´ q 2/(n + 1)ξij with and let Dn = s(A2n ) − [s(An )]2 = Sn [δ(Sn A2n Sn ) − (δ(Sn An Sn ))2 ]Sn . Then we have, for all k = 1, . . . , n, µ [δ(Sn A2n Sn )]kk = [δ((Sn An )(Sn An )T )]kk = 2 n+1 ¶X n p=1 n X 2 apq ξqk q=1 and µ 2 [(δ(Sn An Sn )) ]kk = 2 n+1 ¶X n X n 2 µ ξkp apq ξqk = p=1 q=1 2 n+1 ¶2 n X n X 2 apq ξkp ξqk . p=1 q=1 Hence the kth eigenvalue of Dn is given by ´ ³ Pn 2 p=1 n+1 ³ ´P n 2 p=1 n+1 λk (Dn ) = = ³ = 2 n+1 ³P n q=1 apq ξqk n q=1 apq ξqk − 2 n+1 ³P ´P n 2 p=1 cpk ³ ´2 ´2 − − ´2 P n ( ³ ´2 2 ³ n+1 ´2 2 n+1 ³P n p=1 ³P Pn q=1 apq ξkp ξqk Pn n p=1 ( q=1 ´2 apq ξqk )ξkp ´2 2 p=1 cpk ξkp ) P where cpk = nq=1 apq ξqk . Then by the Cauchy-Schwartz inequality and the orthogonal property of Sn , i.e., n X ( ξip ξpj = p=1 0 n+1 2 for i 6= j, for i = j, we have ³ 2 n+1 ´2 ³P n p=1 cpk ξkp ´2 ≤ = = ´ ´ ³P ´2 ³P n n 2 2 2 · ξ c p=1 kp p=1 pk ³ n+1 ´2 P ³ ´ n 2 n+1 2 c 2 ³ n+1 ´ P p=1 pk n 2 2 p=1 cpk . n+1 ³ Sine Transform Operator 6 Therefore, µ 2 ¶X µ ¶2 X n n 2 2 2 λk (Dn ) = cpk − cpk ξkp ≥ 0, n + 1 p=1 n+1 p=1 Thus Dn = s(A2n ) − [s(An )]2 is positive semidefinite. k = 1, . . . , n. 2 Next, we are going to give some geometric properties of the sine transform operator. For all An , Cn ∈ Fn×n , let hAn , Cn iF ≡ ( n1 )tr(An Cn ). Obviously, hAn , Cn iF is an inner product in Fn×n and hAn , An iF = ( n1 )kAn k2F . We show below that An − s(An ) is perpendicular to Bn×n . Theorem 2.2. Let An ∈ Fn×n , then we have • (i) hAn − (An ), Bn iF = 0, ∀ Bn ∈ Bn×n . • (ii) hAn , s(An )iF = n1 ks(An )k2F . • (iii) kAn − s(An )k2F = kAn k2F − ks(An )k2F . Proof: For (i), ∀ Bn ∈ Bn×n , we have hAn − s(An ), Bn iF = (1/n)tr([An − s(An )]Bn ) = (1/n)tr(An Bn ) − (1/n)tr(s(An )Bn ). Then, we want to prove that tr(s(An )Bn ) = tr(An Bn ). Since Bn = Sn Λn Sn , we have by using (4), tr(s(An )Bn ) = = = = = = = tr[s(An )(Sn Λn Sn )] tr[Sn δ(Sn An Sn )Sn · (Sn Λn Sn )] tr[Sn δ(Sn An Sn )Λn Sn ] tr(δ(Sn An Sn )Λn ) tr((Sn An Sn )Λn ) tr(An (Sn Λn Sn )) tr(An Bn ). Now, the proof of (ii) follows directly from (i). By using (i) and (ii), one can prove (iii) easily. 2 Sine Transform Operator 3 7 The Spectral Properties of Operator s In this section, we discuss some spectral properties of the sine transform operator s. The proof of the following Lemma is similar to that of Theorem 1 in Chan, Jin and Yeung [4] or that of Theorem 2 in Huckle [11]. Lemma 3.1. For An ∈ Fn×n , we have λmin (An ) ≤ λmin (s(An )) ≤ λmax (s(An )) ≤ λmax (An ) where λmax (·) and λmin (·) denote the largest and the smallest eigenvalues respectively. In particular, ks(An )k2 ≤ kAn k2 (5) and if An is positive definite, then s(An ) is also positive definite. For the spectrum properties of the operator s, we have Theorem 3.1. For all n ≥ 1, we have • (i) kskF ≡ sup ks(An )kF = 1. kAn kF =1 • (ii) ksk2 ≡ sup ks(An )k2 = 1. kAn k2 =1 Proof: To prove (i), we notice that if An = (1/n)I, then ks(An )kF = (1/n)kIkF = 1. For general An in Fn×n , by Lemma 2.3 (iii), we have ks(An )k2F = kAn k2F − kAn − s(An )k2F ≤ kAn k2F . Thus ks(An )kF ≤ kAn kF . Hence kskF = 1 for all n. To prove (ii), we have by (5), ks(An )k2 ≤ kAn k2 for all An in Fn×n . Since ks(I)k2 = kIk2 = 1, we have ksk2 = 1 for all n. 2 Sine Transform Operator 4 8 The Numerical Results In this section, we use the PCG method (see [9]) with the optimal sine transform preconditioner s(An ) to solve symmetric Toeplitz system An (f )x = b. Here f is the generating function of the system. We compare the s(An ) with the optimal circulant preconditioner c(An ) which is defined in [8]. Three systems with different generating functions were tested. The generating functions are (i) f (θ) = θ6 + 1, (ii) f (θ) = |θ|3 , (iii) f (θ) = θ4 + (sin θ)2 . When the PCG method is applied to such kind of systems, the stopping criteria for the method is set at krk k2 < 10−7 kr0 k2 where rk is the residual vector at the kth iteration. The right hand side vector b is chosen to be the vector of all ones and the zero vector is the initial guess. Table 1 shows the number of iterations required for convergence and I means no preconditioner was used. Figures 1, 2, 3 show the relative errors at each iteration for different systems with n = 256. f (θ) n 32 64 128 256 512 1024 I 28 59 106 168 203 218 θ6 + 1 |θ|3 s(An ) c(An ) I s(An ) 10 15 22 9 9 13 59 10 7 11 176 11 6 9 562 13 6 7 1934 14 6 8 >2000 15 θ4 + (sin θ)2 c(An ) I s(An ) c(An ) 13 24 8 21 18 55 8 17 25 142 8 18 36 334 7 19 83 752 7 18 190 1625 7 18 Table 1: Number of iterations for different systems. Sine Transform Operator 9 1 10 0 10 −1 2−Norm of the residual 10 −2 10 +++++ No preconditioner −3 −−−−−−−−− Sine transform 10 −4 _____ Circulant 10 −5 10 −6 10 −7 10 −8 10 0 5 10 15 20 25 Number of iterations 30 35 40 Figure 1: Relative errors at each iteration for the system with (i). 4 10 2 2−Norm of the residual 10 0 10 −2 10 −4 10 −6 10 −8 10 0 5 10 15 20 25 Number of iterations 30 35 40 Figure 2: Relative errors at each iteration for the system with (ii). 1 10 0 10 −1 10 −2 2−Norm of the residual 10 −3 10 −4 10 −5 10 −6 10 −7 10 −8 10 −9 10 0 5 10 15 20 25 Number of iterations 30 35 40 Figure 3: Relative errors at each iteration for the system with (iii). Sine Transform Operator 5 10 An Application in Image Restoration In this section, we consider the application of the sine transform preconditioner proposed in previous sections for solving the linear systems arising from image restoration. The mathematical model of the linear image restoration problem is given as follows, see [10], g(ξ, δ) = Z ∞ Z ∞ −∞ −∞ t(ξ − α, δ − β)f (α, β)dαdβ + η(ξ, δ) (6) where g(ξ, δ) is the recorded (or degraded) image, f (α, β) is the ideal (or original) image, the vector η(ξ, δ) represents additive noise. The function t is called the point spread function (PSF) and represents the degradation of the image. Since the PSF here is a function t of ξ − α and δ − β, the t is said to be spatially invariant. The integral in (6) is a two-dimensional convolution. In the digital implementation of (6), the integral is discretized by using some quadrature rule, to obtain the discrete scalar model g(i, j) = m X n X t(i − k, j − l)f (k, l) + η(i, j). k=1 l=1 In matrix form, we have the following linear algebraic system of the image restoration problem, g = Tmn f + η (7) where g, η and f are n2 -vectors and Tmn is an n2 -by-n2 block-ToeplitzToeplitz-block matrix. This is the square image formulation. The PCG method is proposed as main tool to solve (7), see [3] and [5]. We will use the optimal block sine transform preconditioner s(Tmn ) which is defined to be the minimizer of kTmn − Bmn kF over all Bmn ∈ Bmn×mn where Bmn×mn is defined by (2). The following example is constructed. First, we generate an original 256 × 256 image of an ocean reconnaissance satellite, see Figure 4 (left). Then we consider the spatially invariant discretized PSF matrix Tmn with Sine Transform Operator 11 the entries given by q ( exp{−800(i − j)2 − 800(k − l)2 }, if (i − j)2 + (k − l)2 < 1/8, ti−j,k−l = 0, otherwise. (8) The matrix for this example is 65536 × 65536. The observed image is constructed by forming the vector g = Tmn f + η where Tmn is defined by (8), f is a vector formed by row ordering the original image. By unstacking the vector g, we obtain the blurred noisy (observed) image, see Figure 4 (right). The noise function η has normal distribution and is scaled such that kηk2 /kTmn f k2 = 10−4 . Our goal is: given g and Tmn , to recover an approximation to the original image f . Since Tmn is ill conditioning, we use the method of Tikhonov regularization, see [5], and then use the PCG method with the optimal block sine transform preconditioner s(Tmn ). We compare the number of iterations required for convergence of the s(Tmn ) with the optimal block circulant preconditioner c(Tmn ) which is defined in [3] and [5]. The stopping criteria is krk k2 < 10−4 . The convergence results for the image restoration with no kr0 k2 preconditioner Imn , our preconditioner s(Tmn ), and preconditioner c(Tmn ) require 221, 67 and 91 iterations respectively, see Figures 5, 6 and 7(right). 6 Acknowledgment We would like to thank Prof. Raymond H. Chan from the Chinese University of Hong Kong and Dr. C. K. Wong from the University of California at LosAngles for their very helpful suggestions and guidance in the preparation of this paper. Sine Transform Operator 12 Figure 4: Original image (left) and observed image (right). Figure 5: Restored Images with Imn : 20 (left) and 221 iterations (right). Figure 6: Restored Image with s(Tmn ): 20 (left) and 67 iterations (right). Figure 7: Restored Image with c(Tmn ): 20 (left) and 91 iterations (right). Sine Transform Operator 13 References [1] E. Boman and I. Koltracht, Fast transform based preconditioners for Toeplitz equations, SIAM J. Matrix Anal. Appl., 16:628-645 (1995). [2] R. Chan, Circulant preconditioners for Hermitian Toeplitz systems, SIAM J. Matrix Anal. Appl., 10:542-506 (1989). [3] R. Chan and X. Jin, A Family of Block Preconditioners for Block System, SAIM J. Sci. Statist. Comput., 13:1218-1235 (1992). [4] R. Chan, X. Jin and M. Yeung, The circulant operator in the Banach algebra of matrices, Linear Algebra Appl., 149:41-53 (1991). [5] R. Chan, J. Nagy and R. Plemmons, FFT-Based Preconditioners for Toeplitz-Block Least Squares Problems, SIAM J. Sci. Numer. Anal., 30:1740-1768 (1993). [6] R. Chan, M. Ng and C. Wong, Sine Transform Based Preconditioners for Symmetric Toeplitz Systems, Linear Algebra Appl., 232:237-259 (1996). [7] R. Chan and C. Wong, Sine transform based preconditioners for elliptic problems, Numer. Linear Algebra Appls., to appear. [8] T. Chan, An optimal circulant preconditioner for Toeplitz systems, SIAM J. Sci. Statist. Comput., 9:766-771 (1988). [9] H. Golub and C. Van Loan, Matrix Computation, 2nd Edition, Johns Hopkins University Press (1989). [10] R. Gonzalez and R. Woods, Digital Image Processing, AddisonWesley Publishing Company, Inc. (1992). [11] T. Huckle, Circulant and skew-circulant matrices for solving Toeplitz matrix problems, SIAM J. Matrix Anal. Appl., 13:767-777 (1992). [12] T. Huckle, Fast transform for tridiagonal linear equations, BIT, 34:99112 (1994). Sine Transform Operator 14 [13] X. Jin, Sine transform preconditioners for second-order partial differential equations, Numerical Mathematics - A Journal of Chinese Universities, 2:116-123 (1993). [14] X. Jin, A Note on Preconditioned Block Toeplitz Matrices, SIAM J. Sci. Comput., 16:951-955 (1995). [15] E. Tyrtyshnikov, Optimal and super-optimal circulant preconditioners, SIAM J. Matrix Anal. Appl., 13:459-473 (1992). [16] P. Yip and K. Rao, Fast decimation-in-time algorithms for family of discrete sine and cosine transforms, Circuits Systems Signal Process., 3:387-408 (1984).