Download Control-based Quality Adaptation in Data Stream Management

Control-based Quality Adaptation
in Data Stream Management
Systems (DSMS)
Data Stream Management System
(DSMS)
Financial analysis
Mobile services
Sensor networks
Network monitoring
More …
ta
User
Data
Query
Results
Data
User
Data
DSMS
ta
–
–
–
–
–
Da
Da
• Continuous data,
discarded after being
processed
• Continuously answers
queries
• Applications
User
DSMS architecture
• Network of query
operators (1 – 12)
• Each operator has its
own queue
• Scheduler decides
which operator to
execute
• Query results pushed
to clients
• For our purposes,
DSMS can be viewed
as a blackbox
S1
I
1
4
8
11
5
Data
Streams
II
2
6
S2
III
3
Load
Adaptor
7
9
10
Data Input
Rate
Query
Engine
Output
Streams
12
Average
Delay
DSMS
Load Shedding
• Eliminating excessive load by dropping data items 
less QoS violations
• Basic algorithm (Tatbul et al., 2003):
• Key questions
– When?
– How much?
– Where?
• Current solutions focus
on steady-state
performance
Load
What’s missing?
CPU
capacity
– Open-loop control ?
• Assuming there inputs
reach steady states
• However, arrivals are
bursty in practice –
always in transient state
• The solution: closed-loop
control
Time
Why Closed-Loop Control
di
r
1/a
y rr
do
a+dm
1
d m  (a  d m )d i  d o
a
di
y
r
_
y
K
do
a+dm
y
K (a  d m )
(a  d m )
1
r
di 
do
1  K (a  d m )
1  K (a  d m )
1  K (a  d m )
r
1
1
di  d o
K
K
• Reduce the effects of modeling error, input and output
disturbances
• Improve dynamic response
• Stabilize unstable systems
Identification of Database System
Control of Database System
fin
_ fout
1/S
q
y
Cost
Factor
_
Control
• Output is delay time
• Incoming flow rate fluctuates and unknown
• Uncertainties in cost factor
r
Experiments
• Implemented a controller in a real DSMS – Borealis
• With bursty synthetic and real data