Download Document

Exact Recovery of Low-Rank Plus Compressed
Sparse Matrices
Morteza Mardani, Gonzalo Mateos and Georgios Giannakis
ECE Department, University of Minnesota
Acknowledgments: MURI (AFOSR FA9550-10-1-0567) grant
Ann Arbor, USA
August 6, 2012
1
Context
Occam’s razor
Image processing
Among competing hypotheses,
pick one which makes the fewest
assumptions and thereby offers
the simplest explanation of the
effect.
Network cartography
x 10
7
4
|xf,t|
3
2
1
0
0
100
200
300
Time index (t)
400
500
2
Objective
F
T
L
Goal: Given data matrix
and low rank
and compression matrix
, identify sparse
.
3
Challenges and importance
 Both rank(
) and support of
 Seriously underdetermined
generally unknown
LT
X
+
FT
A
(F ≥ L)
>>
LT
Y
 Important special cases
 R = I : matrix decomposition [Candes et al’11], [Chandrasekaran et al’11]
 X = 0 : compressive sampling [Candes et al’05]
 A = 0: (with noise) PCA [Pearson’1901]
4
Unveiling traffic anomalies
 Backbone of IP networks
 Traffic anomalies: changes in origin-destination
(OD) flows
 Failures, transient congestions, DoS attacks, intrusions, flooding
 Anomalies congestion limits end-user QoS provisioning
 Measuring superimposed OD flows per link, identify anomalies
5
Model
 Graph G (N, L) with N nodes, L links, and F flows (F=O(N2) >> L)
(as) Single-path per OD flow zf,t
1
0.9
f2
0.8
0.7
 Packet counts per link l and time slot t
l
0.6
0.5
Anomaly
f1
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
є {0,1}
 Matrix model across T time slots
LxT
LxF
6
Low rank and sparsity
 Z: traffic matrix is7 low-rank [Lakhina et al‘04]  X: low-rank
x 10
4
|xf,t|
3
2
1
0
0
100
200
300
Time index (t)
400
500
 A:
anomaly matrix is sparse across both time
and flows
8
8
x 10
4
|af,t|
|af,t|
4
2
0
0
200
400
600
Time index(t)
800
1000
x 10
2
0
0
50
Flow index(f)
100
7
Criterion
 Low-rank  sparse vector of SVs  nuclear norm || ||* and l1 norm
(P1)
Q: Can one recover sparse
and low-rank
exactly?
A: Yes! Under certain conditions on
8
Identifiability
Y = X0+ RA0 = X0 + RH + R(A0 - H)
X1
 Problematic cases

but

 For
low-rank and
A1
sparse
,
and r = rank(X0), low-rank-preserving matrices RH
 Sparsity-preserving matrices RH
9
Incoherence measures
ΩR
θ=cos-1(μ)
Ф
 Incoherence among columns of R 
,
 Local identifiability
requires
,
 Exact recovery requires
,
 Incoherence between X0 and R
10
Main result
Theorem: Given and , if every row and column of
k non-zero entries and
, then
imply Ǝ
has at most
for which (P1) exactly recovers
M. Mardani, G. Mateos, and G. B. Giannakis,``Recovery of low-rank plus compressed sparse matrices with
application to unveiling traffic anomalies," IEEE Trans. Info. Theory, submitted Apr. 2012.(arXiv: 1204.6537)11
Intuition
 Exact recovery if
 r and s are sufficiently small
 Nonzero entries of A0 are “sufficiently spread out”
 Columns (rows) of X0 not aligned with basis of R (canonical basis)
 R behaves like a “restricted” isometry
 Interestingly
 Amplitude of non-zero entries of A0 irrelevant
 No randomness assumption
 Satisfiability for certain random ensembles w.h.p
12
Validating exact recovery
50
0.7
0.6
40
R
rank(X ) (r)
 Setup
L=105, F=210, T = 420
R=URSRV’R~ Bernoulli(1/2)
X = V’R WZ’, W, Z ~ N(0, 104/FT)
aij ϵ {-1,0,1} w. prob. {ρ/2, 1-ρ, ρ/2}
0.5
30
0.4
0.3
20
0.2
10
0.1
 Relative recovery error
0.1
0
2.5 4.5 6.5 8.5 10.5 12.5
% non-zero entries ()
13
Real data
 Abilene network data
 Dec. 8-28, 2008
 N=11, L=41, F=121, T=504
---- True
---- Estimated
0.8
[Lakhina04], rank=1
[Lakhina04], rank=2
[Lakhina04], rank=3
Proposed method
[Zhang05], rank=1
[Zhang05], rank=2
[Zhang05], rank=3
0.6
0.4
0.2
0
0
0.2
0.4
0.6
0.8
False alarm probability
6
1
Anomaly amplitude
Detection probability
1
5
4
3
2
1
0
Pf = 0.03
Pd = 0.92
Qe = 27%
100
500
400
300
Flow 50
0
Data: http://internet2.edu/observatory/archive/data-collections.html
100
0
200
Time
14
Synthetic data
 Random network topology
1
0.8
0.6
 N=20, L=108, F=360, T=760
 Minimum hop-count routing
0.4
0.2
0
0
0.2
0.4
0.6
0.8
1
1
Detection probability
0.8
---- True
---- Estimated
0.6
0.4
PCA-based method, r=5
PCA-based method, r=7
PCA-based method, r=9
Proposed method, per time and flow
0.2
0
0
0.2
0.4
0.6
False alarm probability
0.8
1
Pf=10-4
Pd = 0.97
15
Distributed estimator
 Centralized
(P2)
 Network: undirected, connected graph
?
?
? ?
?
?
?
n
?
 Challenges
 Nuclear norm is not separable
 Global optimization variable A
 Key result [Recht et al’11]
M. Mardani, G. Mateos, and G. B. Giannakis, "In-network sparsity-regularized rank minimization:
Algorithms and applications," IEEE Trans. Signal Process., submitted Feb. 2012.(arXiv: 1203.1570)
16
Consensus and optimality
(P3)
Consensus with
neighboring nodes
 Alternating-directions method of multipliers (ADMM) solver
 Highly parallelizable with simple recursions
 Low overhead for message exchanges
n
Claim: Upon convergence attains the global optimum of (P2)
17
Online estimator
 Streaming data:
(P4)
ATLA--HSTN
CHIN--ATLA
5
4
0
DNVR--KSCY
20
10
0
HSTN--ATLA
20
Anomaly amplitude
Link traffic level
2
0
WASH--STTL
40
20
0
WASH--WASH
30
20
10
10
0
0
Time index (t)
---- estimated
---- real
0
1000
2000
3000
4000
5000
6000
Time index (t)
o---- estimated
---- real
Claim: Convergence to stationary point set of the batch estimator
M. Mardani, G. Mateos, and G. B. Giannakis, "Dynamic anomalography: Tracking network anomalies via
18
sparsity and low rank," IEEE Journal of Selected Topics in Signal Processing, submitted Jul. 2012.