Download MLSP2012 Tutorial: Manifold Learning: Modeling and Algorithms

Discriminative Recurring Signal
Detection and Localization
Zeyu You, Raviv Raich*, Xiaoli Z. Fern, and Jinsub Kim
School of EECS, Oregon State University, Corvallis, OR 97331-5501
Organization
 Introduction
 Related work and our contribution
 Motivation
 Problem formulation
 Maximum likelihood estimation(MLE)
 Synthetic data experimental Results
 Real-world data experimental Results
Introduction
 What is a recurring pattern?
 DNA motifs
 Music motifs
 Home appliance activations
 Pattern characteristics:
 Sharing same structure
 Recurring in nature
Applications
D'haeseleer, Patrik. "How does DNA sequence
motif discovery work?." Nature
biotechnology 24.8 (2006): 959-961.
Motifs
From Fee Lab Research in
http://web.mit.edu/feelab/research.html
153
154
155
155
152
153
154
154
151
152
152
151
150
0
150
149
4
6
time in (s)
8
10
147
0
151
150
2
4
6
time in (s)
8
10
148
0
152
151
150
149
148
2
153
voltage in (v)
153
voltage in (v)
156
voltage in (v)
voltage in (v)
Air-conditioning activation signals
149
2
4
6
time in (s)
8
10
From Pecan Street dataset (Source: Pecan Street Research Institute)
148
0
2
4
6
time in (s)
8
10
Related work and our
contribution
 Previous works [1-5] focus on:
 discover recurrent patterns from data
 finding the fundamental characteristics of the
signal pattern
 Our contribution:
 a novel formulation of auto-detecting recurring
signal patterns
 a maximum likelihood estimation (MLE) solution
for the problem
 an increased detection performance on a realworld data
Generative vs. Discriminative
Motivation for our approach
 Flavor in discriminative for two reasons
 Robust to variations with in the pattern
 Robust to low signal to noise ratio
Problem formulation
 System diagram:
y(t)
w(t)* x(t)
x(t)
w
LR
Signal
Labeler
 System description:
 Observed data: a collection of M signals
 Hidden data:
 System target:
 To learn a convolutional kernel w.
Y
The graphical model
Instance
labeler
xmt
Signal
labeler
ymt
Ym
T
M
w
The probabilistic model
 Instance labeler (logistic regression):
 Signal labeler:
 Condition model:
The data likelihood
 Data likelihood:
Data
distribution
 Maximum likelihood estimation (MLE):
Independence
 Minimizing the negative log likelihood:
Difference
of convex
Convex-concave procedure
(CCCP)
 Procedure:
 Solution with CCCP [6]:
 Upper bound function (linearization):
 Gradient descent:
Prior Posterior
Synthetic data experiment
 Setup:




Train on M=160;
Test on 40;
Setting kernel size to be F=10, T0=7;
10 MC runs of different initialization.
 Data generation:
 Generate a rectangular pattern;
 Create an empty spectrogram with F=10, T=50;
 Random placing the pattern with varying magnitude into one
time index out of 50;
 Add gaussian noise.
Synthetic results
 Discriminative vs. generative approach:
True pattern
Learned kernel
Discriminative
localization
ROC
1
0.8
True Positive Rate
Generative
localization
Data
0.6
0.4
0.2
discriminative
generative
0
0
0.2
0.4
0.6
False Positive Rate
0.8
1
Real world experiment
 Setup:
 Four home ps-025,029,046,051, 25 days of disaggregated, timesampled electricity usage data from the Pecan Street dataset
({Source: Pecan Street Research Institute})
 Training period 11/17/2012-11/25/2012 meter reading;
 Test period 11/26/2012-12/11/2012;
 Validating kernel size and compared with general approach with
window size set to be T0=700;
 Data generation:
 Extract activations based on power ground truth;
 Extract negative data by random selecting the time where the
power has no significant increase;
 Remove DC offset and Despike large spike noise by median filter;
Fridge activation and data sample
Experiment results
Tuning T0
1
0.9
0.8
0.7
0.6
ps025-air
ps029-furnace
ps046-fridge
ps051-oven
0.5
0.4
Generative detection
0
500
1000
1500
2000
Discriminative detection
1500
50
1000
0
500
-50
0
-100
-500
-1000
-150
0
500
1000
1500
2000
2500
3000
3500
0
500
1000
1500
2000
2500
3000
3500
Detection accuracy
 Performance:
 Discriminative is
better at localization
 Discriminative is more
Invariant to the slight
variations of activation
signals
 Discriminative has
higher AUC than
generative in general
AUC table for both generative and discriminative
Discussion
Can we extent our model to multi-class to
give more discrimination between different
activation patterns?
Can we speedup the algorithm by
converging quicker?
Can we find more applicable real-world
application areas for the algorithm?
References
 [1] Zeyu You, Raviv Raich, and Yonghong Huang, “An inference framework for
detection of home appliance activation from voltage measurements,” in 2014 IEEE
International Conference on Acoustics, Speech and Signal Processing (ICASSP).
IEEE, 2014, pp. 6033–6037.
 [2] Alex S Park and James R Glass, “Unsupervised pat- tern discovery in speech,”
IEEE Transactions on Audio, Speech, and Language Processing, vol. 16, no. 1, pp.
186–197, 2008.
 [3] Aline Cabasson and Olivier Meste, “Time delay estimation: a new insight into the
woody’s method,” IEEE signal processing letters, vol. 15, pp. 573–576, 2008.
 [4] Yoshiki Tanaka, Kazuhisa Iwamoto, and Kuniaki Uehara, “Discovery of timeseries motif from multi-dimensional data based on mdl principle,” Machine Learning,
vol. 58, no. 2-3, pp. 269–300, 2005.
 [5] Jessica Lin, Eamonn Keogh, Stefano Lonardi, and Pranav Patel, “Finding motifs
in time series,” in Proc. of the 2nd Workshop on Temporal Data Mining, 2002, pp. 53–
68.
 [6] Alan L Yuille and Anand Rangarajan, “The concave- convex procedure (cccp),”
Advances in neural information processing systems, vol. 2, pp. 1033–1040, 2002.
Questions?
Thank You!