Download An Overview of Compressed sensing

What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
An Overview of Compressed sensing
M. Vidyasagar FRS
Cecil & Ida Green Chair, The University of Texas at Dallas
[email protected], www.utdallas.edu/∼m.vidyasagar
Distinguished Professor, IIT Hyderabad
[email protected]
Ba;a;ga;va;tua;l .=+a;ma mUa;a;tRa and Ba;a;ga;va;tua;l Za;a:=+d;a;}ba Memorial Lecture
University of Hyderabad, 16 March 2015
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Outline
1
What is Compressed Sensing?
2
Main Results
3
Construction of Measurement Matrices
Deterministic Approaches
Probabilistic Approaches
A Case Study
4
Some Topics Not Covered
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Outline
1
What is Compressed Sensing?
2
Main Results
3
Construction of Measurement Matrices
Deterministic Approaches
Probabilistic Approaches
A Case Study
4
Some Topics Not Covered
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Compressed Sensing: Basic Problem Formulation
Suppose x ∈ Rn is known to be k-sparse, where k n. That is,
|supp(x)| ≤ k n, where supp denotes the support of a vector.
However, it is not known which k components are nonzero.
Can we recover x exactly by taking m n linear measurements of
x?
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Precise Problem Statement: Basic
Define the set of k-sparse vectors
Σk = {x ∈ Rn : |supp(x)| ≤ k}.
Do there exist an integer m, a “measurement matrix” A ∈ Rm×n ,
and a “demodulation map” ∆ : Rm → Rn , such that
∆(Ax) = x, ∀x ∈ Σk ?
Note:
Measurements are linear but demodulation map could be
nonlinear.
The algorithm is universal – the same A and ∆ need to work
for every vector x, and nonadaptive – one has to choose all m
rows of A at the outset.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Further Considerations
What if:
The vector x is not exactly sparse, but only nearly sparse?
The measurement is corrupted by noise, and equals Ax + η
The vector x is (nearly) sparse in some other basis, and not
the canonical basis (e.g., time and frequency)?
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Some Definitions
Given a norm k · k on Rn , for each integer k < n, define
σk (x, k · k) := min kx − zk,
z∈Σk
the k-sparsity index of the vector x.
x ∈ Σk (x is k-sparse) if and only if σk (x, k · k) = 0 for every
norm.
If x 6∈ Σk , then σk (x, k · k) depends on the specific norm.
The k-sparsity index w.r.t. to an `p -norm is easy to compute.
Let Λ0 denote the index set of the k largest components by
magnitude of x. Then
σk (x, k · kp ) = kxΛc0 kp .
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Precise Problem Statement: Final
Given integers n, k n and a real number > 0, do there exist an
integer m, a matrix A ∈ Rm×n , and a map ∆ : Rm → Rn such
that
k∆(Ax + η) − xk2 ≤ C1 σk (x, k · kp ) + C2 ,
whenever η ∈ Rm satisfies kηk2 ≤ ? Here C1 , C2 are “universal”
constants that do not depend on x or η.
If so the pair (A, ∆) is said to display near-ideal signal recovery.
This formulation combines several desirable features into one.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Interpretation
Suppose x is k-sparse, and suppose y = Ax + η. If an “oracle”
knows the support set of x, call it J, then the standard
least-squares estimate is
x̂ = (AtJ AJ )−1 AtJ y = x + (AtJ AJ )−1 AtJ η,
kx̂ − xk2 = k(AtJ AJ )−1 AtJ ηk2 ≤ C0 kηk2 ,
where C0 is the induced norm of (AtJ AJ )−1 AtJ . With a pair
(A, ∆) that achieves near-ideal signal recovery, we have
kx̂ − xk2 ≤ C2 kηk2 .
So, if (A, ∆) achieves near-ideal signal recovery, then the
estimation error is a constant multiple of that achievable by an
“oracle,” but without knowing the support set of x.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Interpretation (Cont’d)
If x is not sparse, but measurements are noise-free (η = 0), then
the estimate x̂ = ∆(Ax) satisfies
kx̂ − xk2 ≤ C1 σk (x, k · kp ).
So the estimate is within a “universal constant” times the
k-sparsity index of x.
If x is k-sparse and measurements are noise-free, we get exact
signal recovery.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Outline
1
What is Compressed Sensing?
2
Main Results
3
Construction of Measurement Matrices
Deterministic Approaches
Probabilistic Approaches
A Case Study
4
Some Topics Not Covered
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Restricted Isometry Property
Definition
A matrix A ∈ Rm×n is said to satisfy the restricted isometry
property (RIP) or order k with constant δk if
(1 − δk )kuk22 ≤ kAuk22 ≤ (1 + δk )kuk22 , ∀u ∈ Σk .
If J ⊆ {1, . . . , n} and |J| ≤ k, then the spectrum of AtJ AJ lies in
the interval [1 − δk , 1 + δk ].
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Candeès-Tao Result on `1 -Norm Minimization
Theorem
Suppose
√ A satisfies the RIP of order 2k with constant
δ2k < 2 − 1. Given x ∈ Rn , define the demodulation map ∆ by
∆(y) = x̂ := arg min kzk1 s.t. Az = y.
z
Then
∆(Ax) = x, ∀x ∈ Σk .
In plain English, any k-sparse vector can be recovered exactly by
minimizing kzk1 for z in A−1 (Ax). Note that this is a linear
programming problem.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
A More General Result
Theorem
m×n satisfies the RIP of order 2k with constant
Suppose
√A∈R
δ2k < 2 − 1, and that y = Ax + η where kηk2 ≤ . Define
x̂ = arg min kzk1 s.t. ky − Azk2 ≤ .
z
Then
kx̂ − xk2 ≤ C0
σk (x, k · k1 )
√
+ C2 ,
k
where
√
√
1 + ( 2 − 1)δ2k
4 1 + δ2k
√
√
C0 = 2
, C2 =
.
1 − ( 2 + 1)δ2k
1 − ( 2 + 1)δ2k
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Some Observations
Bounds are valid for all vectors x ∈ Rn – no assumptions of
sparsity.
The smaller we can make the RIP constant δ, the tighter the
bounds.
We suspect that making δ smaller requires larger values of m
(more measurements).
Indeed this is so, as we shall discover next.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Outline
1
What is Compressed Sensing?
2
Main Results
3
Construction of Measurement Matrices
Deterministic Approaches
Probabilistic Approaches
A Case Study
4
Some Topics Not Covered
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Construction of Matrices with the RIP
Challenge: Given integers n (dimension of√the vector), k (desired
level of sparsity), and real number δ ∈ (0, 2 − 1), choose an
integer m and a matrix A ∈ Rm×n such that A has the RIP of
order 2k with constant δ.
Refresher: A matrix A ∈ Rm×n is said to satisfy the restricted
isometry property (RIP) or order k with constant δk if
(1 − δk )kuk22 ≤ kAuk22 ≤ (1 + δk )kuk22 , ∀u ∈ Σk .
Two distinct classes of approaches: deterministic and probabilistic.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Outline
1
What is Compressed Sensing?
2
Main Results
3
Construction of Measurement Matrices
Deterministic Approaches
Probabilistic Approaches
A Case Study
4
Some Topics Not Covered
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Coherence of a Matrix
Given a matrix A ∈ Rm×n , assume w.l.o.g. that it is
column-normalized, that is each column has `2 -norm of one.
Definition
The one-column coherence µ1 (A) is defined as
µ1 (A) := max max |hai , aj i|.
i∈[n] j∈[n]\{i}
The k-column coherence µk (A) is defined as
X
µk (A) := max
max
|hai , aj i|.
i∈[n] S⊆[n]\{i},|S|≤k
j∈S
Coherence quantifies how “nearly orthonormal” the columns are.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Consequence of Low Coherence
Easy consequence of the Gerschgorin circle theorem:
Lemma
Suppose A ∈ Rm×n is column-normalized. Let k ≤ m be a fixed
integer. Suppose S ⊆ {1, . . . , n} and that |S| ≤ k ≤ m. Then
spec(AtS AS ) ∈ [1 − µk−1 (A), 1 + µk−1 (A)].
Consequently A satisfies the RIP of order k with constant
δk = µk−1 (A).
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
A Lower Bound on Coherence
The following result is known as the “Welch bound.”
Lemma
Suppose A ∈ Rm×n is column-normalized; then
r
n−m
1
≈√ ,
µ1 (A) ≥
m(n − 1)
m
r
n−m
k
µk (A) ≥ k
≈√ ,
m(n − 1)
m
√
In view of the Welch bound, any matrix with µk (A) ≈ k/ m can
be thought of as having “optimal coherence.”
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
A Typical Construction
Due to DeVore (2007). Let p be a prime, and let a be a polynomial
over the finite field Z/(p). Define the p × p matrix M (a) by
1 if j = a(i),
[M (a)]ij =
0 if j 6= a(i).
Note that each column of M (a) has one 1 and the rest 0. Define
the p2 × 1 column vector ua by concatenating the p columns of
M (a), and note that ua has exactly p ones and the rest are zero.
2
r+1
Now define A0 ∈ {0, 1}p ×p
by lining up the columns M (a), as
a varies over all pr+1 polynomials of degree r or less with
√
coefficients in Z/(p). Finally, define A = A0 / p.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
A Typical Construction (Cont’d)
Theorem
The matrix A ∈ Rp
2 ×pr+1
has coherence
µk (A) ≤
kr
.
p
Note that p is the square root of the number of rows of A. So
this construction is within a factor of r of an “optimally
coherent” matrix.
The matrix A is very sparse, with only 1/p elements being
nonzero.
√
The nonzero elements are all equal to 1/ p; so this matrix is
“multiplication free.”
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Examples
Given n, k, δ, we need to choose a prime number p such that
(2k − 1)r
(2k − 1)r 1/(r+1)
r+1
≤ δ, n ≤ p
⇐⇒ p ≥ max
,n
.
p
δ
We can choose r as we wish. Then m = p2 .
Example 1: Let n = 10, 000, k = 5, δ = 0.4. Choosing r = 3 gives
p ≥ 67.5 =⇒ p = 71 and m = p2 = 5, 041. Choosing r = 2 gives
p ≥ 45 =⇒ p = 47 and m = p2 = 2, 209.
Example 2: Let n = 106 , k = 10, δ = 0.4. Choosing r = 3 gives
p ≥ 142.5 =⇒ p = 149 and m = p2 = 22, 201. Choosing r = 2
gives p ≥ 100 =⇒ p = 101 and m = p2 = 10, 201.
In general we get m ≈ n2/3 unless k is very large.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Outline
1
What is Compressed Sensing?
2
Main Results
3
Construction of Measurement Matrices
Deterministic Approaches
Probabilistic Approaches
A Case Study
4
Some Topics Not Covered
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Probabilistic Approach
Let X 0 be a random variable with zero mean and standard
√
deviation of one. Define X = X 0 / m, and define Φ ∈ Rm×n to
consist of nm independent realizations of X. Then it is easy to see
that
E[kΦuk22 ] = kuk22 , ∀u ∈ Rn .
If the r.v. kΦuk22 is also “highly concentrated” around its expected
value, then “with high probability” the matrix Φ satisfies the RIP.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Sub-Gaussian Random Variables
A r.v. X is said to be sub-Gaussian if there exist constants α, β
such that
Pr{|X| > t} ≤ α exp(−βt2 ), ∀t > 0.
A normal r.v. satisfies the above with α = 2, β = 0.5.
For later use, define
c=
β2
4α + 2β
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Main Result
Set-up: Given a constant δ, choose any < δ (preferably very
close to δ), and choose any r such that
r
1+
r ≤1−
.
1+δ
Theorem
Suppose X 0 is sub-Gaussian with constants α, β, and define
c = β 2 /(4α + 2β). Define Φ as nm independent realizations of
√
X 0 / m. Then Φ satisfies the RIP of order k with constant δ with
probability at least equal to 1 − ζ, where
ζ=2
en k 3 k
k
M. Vidyasagar FRS
k
exp(−mc2 ).
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Implementation of the Approach
Suppose k, δ are given, and choose , r as above. To ensure that Φ
satisfies the RIP of order k with constant δ with probability
≥ 1 − ζ, it suffices to choose
2
en
3
1
+ k log
m ≥ 2 log + k log
c
ζ
k
r
samples of X. Note that
m≈
k
log n
cδ 2
plus other terms.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Examples
Choose X to be a Gaussian, so that α = 2, β = 0.5, and
c = β 2 /(4α + 2β) = 1/44. Let ζ = 10−6 .
Example 1: Let n = 10, 000, k = 5, δ = 0.4. Then
m = 21, 683 > n. Compare with m = 2, 209 for the deterministic
approach.
Example 2: Let n = 106 , k = 10, δ = 0.4. Then m = 49, 863.
Compare with m = 10, 201 for the deterministic approach.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Some General Considerations
In the deterministic approach, m ≈ n2/(r+1) for some integer r,
whereas in the probabilistic approach, m = O(k log n). But the O
symbol hides a huge constant!
In both cases, the probabilistic approach requires more
measurements than the deterministic approach! Moreover, the
deterministic approach leads to a highly sparse matrix, whereas the
probabilistic approach leads to a matrix where every element is
nonzero, with probability one.
Open Problem: Find a deterministic approach that leads to
m = O(k log n) measurements.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Outline
1
What is Compressed Sensing?
2
Main Results
3
Construction of Measurement Matrices
Deterministic Approaches
Probabilistic Approaches
A Case Study
4
Some Topics Not Covered
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Time vs. Frequency Domain
Whether a signal is “sparse” depends on the basis used. For
example, a vector x denoting time samples of a signal may not be
sparse, but its discrete cosine transform (or discrete Fourier
transform) may be sparse.
The use of the DFT requires measurement matrices with complex
elements, but the theory works just the same.
In particular, suppose M is the n × n discrete cosine transform
matrix which is real and orthogonal, or the n × n discrete Fourier
transform matrix which is complex and unitary.
Transposes of “randomly selected” rows of these matrices satisfy
the RIP.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Signal Reconstruction from Random Samples
This example is due to Cleve Moler (2010).
Suppose
x(t) = sin(1394πt) + sin(3296πt),
and we sample at 40KHz for 0.2 seconds (8,000 samples).
The three frequencies involved (1394 Hz, 3296 Hz and 40,000 Hz)
are not commensurate.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Discrete Cosine Transformation
This signal is not sparse in the time domain! However, it is sparse
in the frequency domain. For this purpose we employ the discrete
cosine transform (dct).
Given a vector x ∈ RN , its discrete cosine transform (dct)
y ∈ RN is given by
y(k) = w(k)
N
X
x(n) cos
n=1
π(2n − 1)(k − 1)
2N
, k = [N ],
where the weight vector w is defined by
 q
1

, k = 1,
qN
w(k)
2

N , k = 2, . . . , N.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Inverse Discrete Cosine Transformation
Given a vector y ∈ RN , its inverse discrete cosine transform
(idct) is given by
x(n) =
N
X
w(k)y(k) cos
k=1
π(2n − 1)(k − 1)
2N
, n = [N ],
where the weight vector is as before.
Both the dct and idct correspond to multiplying the vector by an
orthogonal matrix.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Discrete Cosine Transform of Signal
The dct of x(t) = sin(1394πt) + sin(3296πt) is shown below. It is
highly concentrated around two frequencies, as expected.
Spectra of Original and Approximated Signals
60
Original
Approximated
40
Coefficient
20
0
−20
−40
−60
0
100
200
300
400
M. Vidyasagar FRS
500
600
Frequency
700
800
900
1000
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Reconstruction of Signal from Random Samples
There are 8,000 samples of the signal x(·). Now we will choose
500 samples at random from these 8,000 samples, and use those to
reconstruct the signal.
Note: When we generate 500 integers at random between 1 and
8,000, ony 485 distinct integers result.
The next slide shows some of the samples.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Sampling of Signal at Random Locations
The figure below shows the actual samples and some of the
randomly chosen samples.
Original Signal and Random Samples
2
1.5
Signal Value
1
0.5
0
−0.5
−1
−1.5
−2
0
0.002
0.004
0.006
0.008
Time in Seconds
M. Vidyasagar FRS
0.01
0.012
0.014
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Signal Reconstruction via `1 -Norm Minimization
Let n = 8000, k = 485. Let S denote the index set of the
randomly selected samples; note that |S| = 485 = k. Define D to
be the dct of the n × n identity matrix. Thus the j-th column of
D is the dct of the j-th elementary vector. Define A ∈ Rk×n to
consist of the rows of D corresponding to the randomly selected
samples; that is, A equals the projection of D onto the rows in S.
Finally, let b ∈ Rk denote the randomly selected samples.
We will reconstruct the original signal x ∈ Rn by setting
x̂ = arg min kzk1 s.t. Az = b.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Reconstructed Signal in the Time Domain – 1
The figure below shows the original and reconstructed signal for
small values of t.
Reconstruction of a Signal with 8000 Samples Using 500 Frequencies
2
Original
Reconstructed
Original and Reconstructed Signals
1.5
1
0.5
0
−0.5
−1
−1.5
−2
0
0.005
0.01
0.015
0.02
0.025
Time
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Reconstructed Signal in the Time Domain – 2
The figure below shows the original and reconstructed signal for
slightly larger values of t. The reconstruction is indistinguishable
from the original signal.
Reconstruction of a Signal with 8000 Samples Using 500 Frequencies
2
Original
Reconstructed
Original and Reconstructed Signals
1.5
1
0.5
0
−0.5
−1
−1.5
−2
−2.5
0.025
0.03
0.035
0.04
0.045
0.05
Time
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Deterministic Approaches
Probabilistic Approaches
A Case Study
Reconstructed Signal in the Frequency Domain
The figure below shows the dcts of the original and reconstructed
signals.
Spectra of Original and Approximated Signals
60
Original
Approximated
40
Coefficient
20
0
−20
−40
−60
0
100
200
300
400
M. Vidyasagar FRS
500
600
Frequency
700
800
900
1000
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Outline
1
What is Compressed Sensing?
2
Main Results
3
Construction of Measurement Matrices
Deterministic Approaches
Probabilistic Approaches
A Case Study
4
Some Topics Not Covered
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Group Sparsity
Partition the index set {1, . . . , n} into g disjoint groups
G1 , . . . , G g .
A vector x ∈ Rn is “group k-sparse” if its support contains
elements from very few groups, and |supp(x)| ≤ k.
“Group” analogs of all previous results exist, e.g. group RIP
(GRIP), and both exact recovery of group k-sparse vectors, as well
as approximate recovery of nearly group k-sparse vectors.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Alternatives to the `1 -Norm
What is so special about the `1 -norm?
It turns out: Nothing!
There are infinitely many norms that permit exact recovery of
sparse or group-sparse vectors.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
One-Bit Compressed Sensing
In standard compressed sensing, the measurement vector is
y = Ax, or yi = hai , xi, for i = 1, . . . , m.
What if yi = sign(hai , xi), for i = 1, . . . , m? This is called one-bit
compressed sensing.
The subject is still in its infancy.
Perhaps the problem can be effectively analyzed using probably
approximately correct (PAC) learning theory.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Low Rank Matrix Recovery
Suppose M ∈ Rl×s has low rank, say ≤ k. By randomly sampling
just m ls elements of M , is it possible to recover M exactly?
Results to similar to vector recovery. Norm of a vector is replaced
by “nuclear norm,” which is the sum of the singular values of a
matrix.
M. Vidyasagar FRS
Overview of Compressed Sensing
What is Compressed Sensing?
Main Results
Construction of Measurement Matrices
Some Topics Not Covered
Questions?
M. Vidyasagar FRS
Overview of Compressed Sensing