Download PowerPoint Presentation - Real-time scene change detection on

Real-Time Networking:
Data, Voice, and Video
Dan Schonfeld
Multimedia Communications Laboratory
ECE Department
University of Illinois
Chicago, Illinois
Multimedia Communications
Laboratory
Rashid Ansari
Ashfaq Khokhar
Dan Schonfeld
Oliver Yu
A Vision To The Future!
Real-Time High-Quality Global Video Communications over HighSpeed Networks
Research Areas
Video Networking
Video Tracking
Video Retrieval
Video Networking
Collaborators:
Rashid Ansari (UIC)
Bulent Cavusoglu (UIC)
Tom DeFanti (UIC)
Jason Leigh (UIC)
Emir Mulabegovic (UIC)
Oliver Yu (UIC)
Homeland Security
Application Testbed

Chicago Video Surveillance
 Monitoring of Biochemical Sensor Arrays
 Helmet Mounted Cameras and Portable Sensors
for Search/Rescue Operations
 Dedicated Emergency Telephony Network
– Public network lack of reliability during emergency
Real-Time
Surveillance/Monitoring

Central Monitoring
– Ok as long as links are
dedicated

Search and rescue

Remote sharing with
agencies in the field

Mobile and Portable
Emergency Response
Center
Central
Monitoring
Next Generation Internet
Architecture
Guaranteed & ControlledLoad
Traffic Control
Topology Distribution
Path Selection
IP Routing
BGP
IGP
ICMP
IP QoS Control
MPLS Forwarding
Signaling Protocol
Non-Real-time
User Application
Real-time Multimedia
HTTP/SMTP/FTP/TELNET
Middleware
CR-LDP
RSVP
Signaling Transport
Out-of-Band
Associated
ATM/PPP/HDLC
SONET
Transport
TCP
UDP
RTP
IP
Fiber/DWDM
Real-time
/ MPLS
Network & Layer
Management
Best Effort
Curren
t
Interne
NGI
t
Extensio
n
VIPER: FEC/UDP Over
STARTAP
(Chicago-Amsterdam)
Latency of transmitting 100 packets under
UDP, TCP, FEC/UDP with 3:1 redundancy.
1-way latency in ms
UDP
400
TCP
350
FEC over UDP
300
250
200
150
100
50
0
0
500
1000
1500
Packet size in bytes
2000
2500
APPLICATION
Real-Time
Transport
Protocol
(RTP)
RTP
TRANSPORT
UDP
IP
DATA LINK
PHYSICAL
Picture Type
MPEG-1 Header extension
MBZ
T
TR
AN
N
S
B
E
f_code
X
E
f(0,0)
P
FBV
BFC
FFV
FFC
MPEG-2 Header extension
f(0,1)
f(1,0)
f(1,1)
DC
PS
T
P
C
Q
V
A
R
H
G
D
50
AFEC
staticIPB
staticFEC
optimalFEC
48
44
42
PSNR in dB
Adaptive
FEC
46
40
38
36
34
32
30
1
2
ADAPTIVE
FEC
ENCODER
MPEG-2
RTP
RTP MEDIA
PACKETS
3
4
5
6
7
Packet Loss Ratio Percentage
8
9
10
RTP FEC
PACKETS
NETWORK
RECEIVER
FEC Simulations
(a) Static FEC
(c) AFEC
(b) Static IPB
(d) Original
DiffServ
28
27
Motion
IPB
Greedy
Optimum
26
25
PSNR in dB
24
23
22
21
20
19
18
5.7
5.75
5.8
5.85
5.9
5.95
Mbps
6
6.05
6.1
6.15
DiffServ Simulations
(a) Greedy
(c) Motion
(b) IPB
(d) Original
Rate-Controller
z-1
inv()
( mk )
-1
ak
m k +1
inv()
a k-1 -
1 - a k-1
+
+
1
-1 )
(1 - am
k ) pk +1 (
k
( m k +1 Î μ )
uk
)
pk +1 ( m k )
)
pk +1 (μ)
)
pk ( m k )
pk
Network
pk
uk
)
m k +1
)
choose pkd++1 ( m k 1 )
vk
wk
Kalman
Predictor
z-1
Generate
uk (μ)
4
5
1
5
7
4
4
2
6
7
5
2
4
zk
Rate-Control Simulations
(a) Choke-Only
(b) Choke-ORCA
Lightweight Streaming
Protocol (LSP)
Nominal
frame rate
(NFR)
Sender
Receiver
Framer
Actual
frame rate
(AFR)
Discriminator
Packetizer
Packet
buffer
Sender
To client
RTX
SEQ1
Video
file
Shared
parameters
and statistics
SEQ2(lost)
SEQ3
SEQ4(lost)
SEQ4(rtx)
NACK2
(ignored)
Receiver
Control
messages
from the
client
Server architecture
NACK2
SEQ5
SEQ2 (rtx)
Control
(NACK2 +
NACK4)
Packets
Transmitted
UDP
LSP
LSP-PMN
DSL
94%
99%
100%
Wi-Fi
95%
96%
98%
Packets received
LSP
PSNR = 18 dBms
PSNR = 33 dBms
QoS Control
Over Wireless &
Core Network
CDMA-Based
Admission &
Scheduling
(Oliver Yu)
Wireless
QoS Control
Core
QoS Control
Internet
QoS Control
BS
Backbone
Gateway
Internet
MSC
Backbone
Backbone
Router
AMDG-CAC
FDWFQ-MAC
AP-CAC
Wireless MAC Protocols
(Khokhar)
Idle
Duty Cycle
0.1
0.09
9.30%
0.08
Normalized Power Consumption
Listen
8.16%
0.07
0.06
0.05
TDMA-W, 50 nodes
TDMA-W, 100 nodes
TDMA-W, 200 nodes
10% S-MAC, 50 nodes
10% S-MAC, 100 nodes
10% S-MAC,200 nodes
0.04
0.03
5.13%
0.02
0.01
0.56%
0
0.1
0.2
0.3
Call Admission
Controller
X
X
Channel
Slot Level Scheduler
0.4
0.5
0.6
Event Arrival Rate
0.7
0.8
0.9
Center for Global Multimedia
Mobile Communications
Internet over Cable
Digital Subscriber Lines
A typical ADSL equipment configuration.
Low-Earth Orbit Satellites
Iridium
(a) The Iridium satellites form six necklaces around the earth.
(b) 1628 moving cells cover the earth.
Globalstar
(a) Relaying in space.
(b) Relaying on the ground.
The 802.16 Physical Layer
The 802.16 transmission environment.
Computer Network
Infrastructure
NU
UIC


Optical Networking
Transport Encoding
& Protocols
 Wired and Wireless
Network Integration
 Circuit and Packet
Switched Network
Deployment
NCS
A
UCS
D
Wireless Communications




Opportunistic
Resource Allocation &
Admission Control
Channel Estimation
Power-Efficient
Wireless Protocols
High-Capacity
Wireless Networks
optical Switch
Wireless
Access
Network
Router
Wireless Access Point
Wireless Terminal
Optical Core
Packet
Data
Network
Wireless Access Network:
• IEEE 802.11 (Year 1)
• IEEE 802.16 (Years 2 & 3)
• IEEE 802.20 (Optional)
Applications & Prototypes









Video Communications
Tele-Education
Natural Event Monitoring
Geosciences Monitoring
Environmental Assessment
Emergency Management
Elderly Care
Medical Diagnosis
Remote Robotic Surgery
Visualization & Devices



High-Resolution
Scalable Displays
High-Resolution
Capture
Interactive Tools
Intelligence Sharing
Real Time Monitoring and Real Time Multimedia Retrieval
and Sharing across the continent
Video Tracking
Collaborators:
Nidhal Bouaynaya (UIC)
Karthik Hariharakrishnan (Motorola Research)
Dan Lelescu (NTT DoCoMo Research)
Josh Meir (NeoMagic)
Magdi Mohamed (Motorola Research)
Wei Qu (UIC)
Philippe Raffy (R2 Technology)
Fathy Yassa (NeoMagic)
Motivation









Target Tracking
Surveillance
Retrieval
Video Coding
Video Communications
Videoconferencing
Virtual Reality
Human-Computer Interaction
Computer Animation
VORTEX
VORTEX: Video Retrieval and Tracking
from Compressed Multimedia Databases
Object cluster
Reference frame
VORTEX
Template
Template Matching [sec] VORTEX [sec]
Object #1
45.22
0.0084
Object #2
39.36
0.0092
Adaptive Block Matching
(ABM)
Method
Time [sec]
ABM
10
Partition Projection
Partition Lattice Operators
165
193
Motion-Based
Particle Filters
Condensation filter
MBPF
Multi-Object Particle Filters
x1
....
x2
p(xt|xt-1)
xt
p(zt|xt)
z1
z2
zt
HMM
x11
...
...
x12
x22
x12
z11
xt2
...
z12
z22
z2m
MHMM
zt2
.
..
z1m
xtm
zt1
.
..
.
..
z12
.
..
.
.
..
..
x2m
x1m
xt1
ztm
Experimental Results
IDMOT
The Dynamic Graphic Model for Multiple
Interactive Objects In Two Frames
Magnetic-Inertia Model
Punish
Reward
Video Tracking and
Foveation
(Ansari & Khokhar)
Future Research:
Video Tracking









Randomly Perturbed Active Surfaces
Video Stabilization
Auto-Focus Recovery
Pose Estimation and Feature Tracking
Video Animation
Stereography from a Single Camera
Multiple Camera Mosaics
Multiple Camera Tracking
Low-Power Particle Filters
Video Retrieval
Collaborators:
Faisal Bashir (UIC)
Ashfaq Khokhar (UIC)
Dan Lelescu (NTT DoCoMo Research)
Fatih Porikli (Mitsubishi Research)
Motivation
• Video Surveillance
• Sign Language Recognition
• Sports Video Analysis
• Animal Mobility Experiments
• Moving Object Databases
• Video and Sensor Databases
Spectral Clustering
Trajectory Retrieval
3000
2500
2000
PCA-Global
PCA-Seg Euc
1500
PCA-Seg Str
1000
Lei Chen
500
0
1
2
3
4
5
6
Gaussian Mixture Models
Nc
P( y  )    i ( y; mi , i )
i 1
Figure: 1-Sigma contours of GMM’s learnt from three classes.
(a) ‘Norway’. (b) ‘Alive’. (c) ‘Crazy’.
Classification
HMM for Class 1
HMM for Class N
Gaussian
Mixtures
…
Classification:
arg max p (Y1 , ,Ym i )
iÎ1, ,L
…
Training Set Database
Accuracy
ASL
#Classes : # Trajectories
Datasets
HJSL
2:138
4:276
8:552
16:1104
29:2001
38:2622
HMM
0.9638
0.9167
0.8587
0.7790
0.6882
0.6609
0.9074
GMM
0.9855
0.8949
0.8514
0.7455
0.6672
0.6400
0.8981
Moghadd
0.9420
am
0.9312
0.8297
0.7283
0.5592
0.6175
0.4537
Accuracy values for various class sizes from ASL data set and the HJSL dataset
(last column). Column headings are shown as (number of classes:number of
trajectories) for the ASL dataset at different sizes.
Shape Representation
Curvature Scale Space
Figure: An example high jump trajectory and its translated, rotated
CSS Imagesscaled
of a Trajectory
itswith
36-degree
rotated
version
and non-uniformly
version, and
along
their CSS
images.
Performance
Indexing Time
(sec.)
(408 Traj.)
Retrieval Time
(sec.)
(15 Traj.)
PCA Centroid
178.9270
8.2920
Hybrid PCA
175.2020
27.5400
CSS
1508.3
28.0500
Future Research:
Video Retrieval

Trajectory Occlusion
 Camera Motion
 Multiple Cameras
 Multiple Trajectories
 Video Mining
 Joint Retrieval, Recognition, & Mining
 Multi-Modality Feature Integration