Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Computational electromagnetics wikipedia , lookup
Exact cover wikipedia , lookup
Perturbation theory wikipedia , lookup
Laplace–Runge–Lenz vector wikipedia , lookup
Knapsack problem wikipedia , lookup
Computational complexity theory wikipedia , lookup
Inverse problem wikipedia , lookup
Linear algebra wikipedia , lookup
Genetic algorithm wikipedia , lookup
Multi-objective optimization wikipedia , lookup
Multiple-criteria decision analysis wikipedia , lookup
1 Equivalence of Reconstruction from the Absolute Value of the Frame Coefficients to a Sparse Representation Problem Radu Balan*, Senior Member, IEEE, Pete Casazza, and Dan Edidin Abstract— The purpose of this note is to prove, for real frames, that signal reconstruction from the absolute value of the frame coefficients is equivalent to solution of a sparse signal optimization problem, namely a minimum `p (quasi)norm over a linear constraint. This linear constraint reflects the coefficients relationship within the range of the analysis operator. Index Terms— frames, nonlinear processing, sparse representation I. I NTRODUCTION In our previous paper [2] we considered the problem of signal reconstruction from the absolute value of its coefficients in a redundant representation. We obtained necessary and sufficient conditions for perfect reconstruction up to a constant phase factor. In the finite dimensional setting, a frame for a Hilbert space is just a set of vectors spanning the Hilbert space. For real valued signals and real valued transformations we obtained the following result Theorem 1.1: Let M : RN /{+1, −1} → RM be defined by M(x) = {|hx, fk i|}1≤k≤M , where F = {f1 , f2 , . . . , fM } spans RN . Then: 1) If M ≥ 2N − 1 then for a generic frame F, the map M is injective; 2) If M is injective, then M ≥ 2N − 1; 3) If M = 2N − 1 then M is injective if and only if every N -element subset of F is linearly independent. 4) M is injective if and only if for every subset G ⊂ F, either G or F \ G spans RN . 5) If M > N then for a generic frame F, the set of points x ∈ RN /{+1, −1} so that M−1 (M(x)) contains one point, is dense in RN . ♦ Here generic frames denote an open and dense set of frames, with respect to the topology induced by the Grassmanian manifold topology (see [2]). In a completely different line of research, Donoho and Huo obtained in their seminal paper [3] an equivalence result for solving sparse optimization problems. More specifically let us *Corresponding Author: Radu Balan is with Siemens Corporate Research,755 College Road East, Princeton NJ 08540, [email protected]. Pete Casazza and Dan Edidin are with Department of Mathematics, University of Missouri, Columbia MO 65211, [email protected] , [email protected] . Manuscript received September 7, 2006; revised XXX, 2006. The second author was supported by NSF DMS 0405376. consider the following objects. R (1) √ 1 (1 + N ) (2) D(N ) = 2 where I is the N × N identity matrix, PM and R is the N -point unitary FFT matrix. Let kckp = ( k=1 |ck |p )1/p for any M vector c, and p > 0. For p = 0, we define kck0 to be the number of nonzero entries of c. With these notations Donoho and Huo showed the following result: Theorem 1.2: Assume x = T c0 for some c0 ∈ C2N so that kck0 < D(N ). Then the following two optimization problems admit the same unique solution c0 : T ĉ0 ĉ1 = I = argminc:x=T c kck0 = argminc:x=T c kck1 (3) (4) ♦ The bound D(N ) as stated here was √ next improved by Elad √ and Bruckstein in [5] to D(N ) = ( 2 − 12 ) N , and also extended to other matrices than this particular T (see also [4], [6], [7]). Optimization problems of type (3) and (4) are also related to sparse multicomponent signal decompositions. More specifically, consider the following estimation problem. Given the model s x = Us + V t = U V (5) t where x ∈ RN is the vector of measurements, s, t ∈ RM are vectors of unknown component coefficients, U, V are known N × M mixing matrices, the problem is to obtain the Maximum A Posteriori (MAP) estimator of the two components U s and V t, when s and t are known to have prior distributions of the form p p pS (s) ∝ exp(−α kskp ) , pT (t) ∝ exp(−β ktkp ) (6) It is then immediate to derive the MAP estimator as: p p (ŝ, t̂)M AP = argminU s+V t=x α kskp + β ktkp (7) Note for p < 1, prior distributions of type (6) have long tails, are peakier than Gaussian (p=2), or even Laplacian (p=1) distributions, and allocate uniformly larger costs for nonzero components compared to the vanishing components. In this short note we present a necessary condition for perfect reconstruction from the absolute value of the frame coefficients in terms of a sparse representation optimization 2 for all 1 ≤ k ≤ M . Note t, s ≥ 0. We claim u = [tT sT ]T and ũ = [sT tT ]T are the only solutions of (10). Since they satisfy F T x = t − s, we obtain a = |t + s|, and x = G(t − s). II. N OTATIONS To prove the claim, first note that both u and ũ are feasible We use the following notations. Let F = [f1 | · · · |fM ] denote the N × M real matrix whose vectors, that is they satisfy the linear constraint Au = α. Let 0 T T T 0 columns are the M frame vectors {f1 , . . . , fM } in RN . Let u = [v w ] be another feasible vector, that is Au = α. 0 0 G = [f˜1 | · · · |f˜M ] denote the N × M matrix whose columns We will prove ku kp ≥ kukp , and ku kp = kukp if and only if either u0 = u, or u0 = ũ. are the canonical dual frame vectors. That is, Indeed, we have u0 = u + d, where d ∈ ker A. Due to the F GT = GF T = IN , (perfect reconstruction) special form of A in (9), we obtain: F T G = GT F = P , (projection onto the coefficients space) h d= , h ∈ Ran P (13) −h Recall the map we are interested in is: problem. Furthermore, the optimization problem gives the solution for the reconstruction problem. M : RN /{+1, −1} → RM , M(x) = {|hx, fk i|}1≤k≤M (8) where RN /{+1, −1} = {x̂ ; } is the set of classes of vectors x̂ obtained by identification of x with −x, that is x̂ = {x, −x}. Perfect reconstruction up to a sign of the vector x is possible if and only if M is injective. Let a ∈ RanM := {|F T x| ; x ∈ RN }. We denote by A, K, and α the 2M × 2M real matrices, respectively the 2M vector, defined by I I 0 I a A= , K= , α= I − P −(I − P ) I 0 0 (9) For vectors u ∈ Rm , u ≥ 0 means uk ≥ 0 for every k. For y ∈ Rm , we denote |y| the vector of absolute values of its entries, |y| = (|yk |)1≤k≤m . We also let supp(y) denote the support of the vector y, that is supp(y) = {k ; yk 6= 0}. III. M AIN R ESULT k∈I0 k∈I1 (10) admits exactly two solutions u and ũ = Ku ∈ R2M independent of p, with u = [tT sT ]T so that a = |t + s|, and x̂ = {G(t − s), −G(t − s)}. Conversely, if for some 0 ≤ p < 1 the optimization problem (10) admits at most two solutions, then M−1 (a) contains exactly one point. ♦ The proof is based on two key ingredients: one is the decomposition of any real number into its positive and negative parts; the other ingredient is the inequality |a+b|p ≤ |a|p +|b|p , for all a, b ∈ R and 0 ≤ p < 1, where the equality holds if and only if one of a, b is zero. Proof (1) Proof of ⇒. Let x ∈ M−1 (a). Thus a = |F T x|. Let t, s be the positive, and respectively the negative part of F T x, that is: (F T x)k if (F T x)k ≥ 0 tk = (11) 0 otherwise 0 if (F T x)k ≥ 0 sk = (12) T −(F x)k otherwise k∈I2 Let σ : {1, 2, . . . , 2M } → {1, 2, . . . , 2M } be the map σ(m) = m + M , if m ≤ M , and σ(m) = m − M for m > M . Let j : {1, 2, . . . , 2M } → {1, 2, . . . , M } be the map j(m) = m, for m ≤ M , and j(m) = m − M , for m > M . Because of special form of d, k0 ∈ J if and only if σ(k0 ) ∈ J, that is σ(J) = J. On the other hand, the choice of t and s guarantees that k0 ∈ I if and only if σ(k0 ) 6∈ I, that is I ∩ σ(I) = ∅. Thus if k ∈ I1 then σ(k) ∈ I2 (although σ(I1 ) ⊂ I2 may be strict) and |dk | = |dσ(k) | = |hj(m) |. Let I21 = σ(I1 ) and I22 = I2 \ I21 . Thus I2 = I21 ∪ I22 , and X X X X p ku0 kp = |uk |p + |uk + dk |p + |dk |p + |dk |p k∈I0 Theorem 3.1: Let a ∈ Ran M. If M−1 (a) contains only one point, say {x̂}, then for every 0 ≤ p < 1 the following optimization problem argmin Au = α kukp Let J = supp(d) = {k ; dk 6= 0} ⊂ {1, 2, . . . , 2M }, I = supp(u), and I0 = I \ J, I1 = I ∩ J, and I2 = J \ I. Then supp(u0 ) ⊂ I0 ∪ I1 ∪ I2 and X X X p ku0 kp = |uk |p + |uk + dk |p + |dk |p (14) p = kukp + X k∈I1 k∈I1 k∈I21 [|uk + dk |p + |dk |p − |uk |p ] + k∈I22 X |dk |p k∈I22 For 0 ≤ p < 1, we have |uk + dk |p + |dk |p ≥ |uk |p with equality achieved if and only if either dk = 0, or dk = −uk . This proves that: ku0 kp ≥ kukp (16) and thus u is a global optimizer for (10). The only remaining issue is to prove that if ku0 kp = kukp , then either u0 = u, or u0 = Ku. Equations (15) prove that ku0 kp = kukp if and only if I22 = ∅ and for all k ∈ I1 , either dk = −uk , or dk = 0. However dk 6= 0 for k ∈ I1 (by construction), hence dk = −uk for all k ∈ I1 . This means: uk f or k ∈ I0 0 uk f or k ∈ I21 = σ(I1 ) (u )k = (17) 0 otherwise Let t0 , s0 be the M -components of u0 , u0 = [(t0 )T (s0 )T ]T . The feasibility constraint Au0 = α implies that t0 + s0 = a, and (I −P )(t0 −s0 ) = 0. Let c0 = t0 −s0 . Thus P c0 = c0 , hence c0 ∈ Ran F T . On the other hand, since I22 = ∅ and σ(I1 ) ⊂ I2 , we have that σ(I1 ) = I2 . Hence, σ(I2 ) = σ 2 (I1 ) = I1 and so I0 ∩σ(I2 ) = I0 ∩I1 = ∅. Also, by construction I0 ∩I2 = ∅ and it follows σ(supp(u0 ))∩supp(u0 ) = ∅. Thus |c0 | = t0 +s0 = a. This means x0 = Gc0 is in M−1 (a). Since M−1 (a) contains (15) 3 only one point, x̂ = {x, −x}, it follows that either u0 = u, or u0 = −u, which ends the proof of this way. (2) Proof of ⇐. Fix a ∈ M, and assume M−1 has at least two classes, say x̂ = {x, −x} and x̂0 = {x0 , −x0 }. Consider again the positive and negative parts of F T x and F T x0 , respectively, say t, s, and t0 , s0 . Since x 6= x0 and x 6= −x0 , yet |F T x| = |F T x0 |, it follows that t0 6= t and t0 6= s, yet kukp = ku0 kp , where u = [tT sT ]T and u0 = [t0T s0T ]T . Thus u, Ku, u0 , Ku0 are distinct solutions of (10), which proves the converse. Q.E.D. ♦ Remark 1: Note that for p = 1 the result does not hold true. Indeed, for p = 1 any feasible vector u of (10) of positive components has the same l1 norm: kuk1 = kak1 . Remark 2: If u is a feasible vector, that is it satisfies Au = α, then Ku is also a feasible vector, that is it satisfies AKu = α, In particular, if u is a solution of (10) then Ku is also a solution of same problem. Remark 3: Note that we do not impose any positivity constraint on u in (10). And yet, remarkably, the optimizer turns out to have nonnegative components. Remark 4: Connections between `p optimization problems and sparse signal representations have been studied in literature in the context of solving ICA type problems; see [8], [9], [1]. Corollary 3.2: If M is injective, then for all a ∈ Ran M, and 0 ≤ p < 1, the optimization problem (10) admits only two solutions u and Ku so that M−1 (a) = {G(t − s), G(s − t)}, where u = [tT sT ]T . Corollary 3.3: If (10) admits at most one solution for all a, then M is injective. IV. C ONCLUSIONS In this paper we study the reconstruction problem of a real signal when only absolute values of its real frame coefficients are known. In general one can expect at most to reconstruct the original signal up to an ambiguity of one global sign. We prove that this is the case if and only if an `p optimization problem, more specifically (10), admits exactly two solutions. Furthermore, the solutions of this problem are directly related to the original (and reconstructed) signal. This result reduces a combinatorial optimization problem (where the combinatorics are due to the exhaustive search over all possible sign combinations) to an `p optimization problem (albeit nonconvex). R EFERENCES [1] R. Balan, J. Rosca, S. Rickard, Equivalence Principle for Optimization of Sparse versus Low-Spread Representations for Signal Estimation in Noise, Int. J. on Imag. Syst. Tech., Vol.15, No.1, pp.10–17, 2005. [2] R. Balan, P.G. Casazza, and D. Edidin, On signal reconstruction without phase, Applied. Comp. Harm. Anal., 20, no.3, pp. 345356, 2006. [3] D. Donoho, X. Huo, Uncertainty Principles and Ideal Atomic Decomposition , IEEE Trans. on Inform. Theory, 47, no.7, pp.28452862, 2001. [4] D. Donoho, M. Elad, Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization, in Proc. Nat.Acad. Sci., 100, no.5, 2197-2202, 2003. [5] M. Elad, A.M. Bruckstein, A Generalized Principle and Sparse Representation in Pairs of Bases, IEEE Trans. on Inform. Theory, 48, no.9, pp. 2558–2567, 2002. [6] J.-J. Fuchs, On sparse representations in arbitrary redundant bases. IEEE Trans. on Inform. Theory, 50, no.6, pp. 1341–1344, 2004. [7] R. Gribonval, M. Nielsen, Sparse Decompositions in Unions of Bases, IEEE Trans. on Information Theory, 49, no.12, pp.3320– 3325, 2003. [8] R. Gribonval, M. Nielsen, On the Strong Uniqueness of Highly Sparse Representations from Redundant Dictionaries, Proc. ICA’04, pp.201-208, Springer-Verlag LNCS Series, September 2004. [9] D.M. Malioutov, M. Cetin, and A.S. Willsky, Optimal Sparse Representations in general overcomplete Bases, Proc. ICASSP ’04, Montreal, Canada , pp. II-793-796 vol.2, May 2004.