Download Event detection - ifp.illinois.edu

Unusual Event Detection
via Multi-camera Video Mining
Hanning Zhou ([email protected]) and Don Kimber ([email protected] )
Step 1: Temporal segmentation
Introduction
 Goal: detecting unusual events from a large
amount of multiple stream video.
 Challenges: multi-stream video
lack of labeled data
 Approach: semi-supervised learning
Applications
static scene
……………………
……
…
…
…………………
Segment k
 Online
 Detect unusual events alarm in surveillance system
 Offline
 Detect highlights from sport videos
 Analyze business process
 Other data streams
 audio, text and multimodal data streams
…
…
Segment k + 1
Step 2: Feature extraction
 Feature:
 size and location of the motion blobs
 clustered into GMMs
 Advantage:
 higher spatial resolution than motion histogram
[Zhang ‘05] [Zhong ‘04]
Key Idea
 New problem: spatial alignment
 Collaborative mining of multiple streams
 Solved with approximate KL-divergence
[Goldberger ‘03]
 Sensor network is prevailing
 Events from different sensors are related
 Two-Stage Training
………
Step 3: Training a Model for usual event
 Unusual events are rare
 Manual labeling is intractable
 1st Stage: Bootstrap


Previous Work
 General Event Detection
 specific event with well-defined model
 supervised statistical learning (DTW, HMM, factor graph)
 Unusual Event Detection
 unsupervised [Zhong & Shi ‘04]
 semi-supervised [Zhang et al. ‘05]
 All above are from single stream
Experimental Results
 Trained on one week’s video recorded in FXPAL
 Tested on another week’s video
 usual events: pass by, pick up print outs
 unusual events: distribute mails, open multiple drawers
to look for stationery, open the cabinet, multiple people
Clustering: keep the large clusters
User feedback: exam the small clusters
 2nd Stage: Train CHMM for usual events
 Inference in CHMM is efficient
O(T(CN)^2) vs O(TN^(2C))
 statistic model to handle variant durations, noisy
observation and asynchrony
 The hidden state depends on 3D location of the objects
 The observations are 2D projection of the objects
 CHMM as a loose stereo
 Dependent chains
The 3D location inferred from
different cameras are RELATED
Exact stereo does not work well, because:
gaps between the views
wide baseline, few correspondences
Examples of usual events
Step 4: Detecting usual event
Examples of unusual events
 Evaluating the likelihood with forward-backward
algorithm [Brand ‘97]
ROC curve of HMM vs CHMM on Terrascope data
 Quantitative experiments on Terrascope data
 48 segments in 4 scenarios
 usual events:
group meeting, natural video sequence
 unusal events:
group exit, intruder, theft, suspicious behavior
Mailroom camera setup
Terrascope dataset camera setup
Courtesy of Christopher Jaynes