Download Powerpoint - Signal Processing in Networking

Multiscale Traffic Processing Techniques
for Network Inference and Control
R. Baraniuk R. Nowak E. Knightly R. Riedi
X. Wang V. Ribeiro S. Sarvotham Y. Tsang
NMS PI meeting
Rice University, SPiN Group
Baltimore
April 2002
Signal Processing in Networking
spin.rice.edu
Effort 1
Chirp Probing
Rice University, SPiN Group
spin.rice.edu
Objective: Reduced complexity, multiscale
link models with known accuracy
Innovative Ideas
Multifractal analysis
Multiplicative modeling
Multiscale queuing
Chirps for probing
Impact
Congestion control
Workload balancing at servers
Dynamical streaming
Pricing on connection basis
Rice University, SPiN Group
spin.rice.edu
Chirp Probe Cross-Traffic Inference
Rice University, SPiN Group
spin.rice.edu
New Ideas
Probing multiple hops
Chirp-Sandwiches to probe routers
“down the path”
IP-tunneling
Monitor packets at intermediate point of path
Verification for Chirping and Sandwiches
Network calculus (max-plus algebra)
Probing buffer at core router
Passive inference
Rice University, SPiN Group
spin.rice.edu
Tech Transfer
- Stanford (SLAC)
Chirps in PingER as freeware for monitoring
- C+ code for sending Chirp Probes
- 10 Msec precision
- Visualization tools for link properties
IP tunneling
- Infrastructure at Rice ready
- CAIDA (chirping as a monitoring tool)
- UC Riverside (Demystify Self-similarity)
- Sprint Labs (queue-sizes at core routers)
Rice University, SPiN Group
spin.rice.edu
Effort 2
Connection-level Analysis
and
Modeling of Network Traffic
Rice University, SPiN Group
spin.rice.edu
Aggregate Traffic at small scales
Auckland Gateway (2000)
Aggregate Bytes per time
• Trace:
– Time stamped headers
x 10
450
– Gaussian
– LRD (high variability)
350
– Non-Gaussian
– Positive process
– Burstiness
Objective :
– Origins of bursts
Rice University, SPiN Group
Gaussian: 1%
Gaussian: 1%
Real traffic:
3% 3%
Real traffic:
2.5
300
99%99%
2
250
200
• Small scales
histogram
3
400
number of bytes
• Large scales
5
1.5
150
Mean
Mean
1
100
0.5
50
0
0
0
0
0.5
1
2000
1.5
40002
2.5
time (1 unit=500ms)
6000
3
x 10
5
Kurtosis - Gaussian : 3
- Real traffic: 5.8
spin.rice.edu
Bursts in the ON/OFF framework
• ON/OFF model
– Superposition of sources
– Connection level model
• Explains large scale variability:
– LRD, Gaussian
– Cause: Costumers
– Heavy tailed file sizes !!
• Small scale bursts:
– Non-Gaussianity
– Conspiracy of sources ??
– Flash crowds ??
(dramatic increase of active sources)
Rice University, SPiN Group
spin.rice.edu
Non-Gaussianity: A Conspiracy?
Load: Bytes per 500 ms
99%
Mean
• The number of active
connections is close to
Gaussian; provides no
indication of bursts in the
load
Number of active connections
99%
Mean
Rice University, SPiN Group
• Indication for:
- No conspiracy of sources
- No flash crowds
spin.rice.edu
Non-Gaussianity: a case study
Typical Gaussian arrival
Typical bursty arrival
(500 ms time slot)
Histogram of load
offered in same time
bin per connection:
Considerable balanced
“field” of connections
10 Kb
(500 ms time slot)
Histogram of load
offered in same time
bin per connection:
One connection
dominates
150 Kb
15 Kb
Rice University, SPiN Group
spin.rice.edu
Non-Gaussianity and Dominance
Systematic study: time series separation
• For each bin of 500 ms:
remove packets of the ONE strongest connection
• Leaves “Gaussian” residual traffic
99%
=
Mean
Overall traffic
Rice University, SPiN Group
+
1 Strongest connection
Residual traffic
spin.rice.edu
Separation on Connection Level
Definition:
• Alpha connections:
Peak rate > mean arrival rate + 1 std dev
• Beta connections: Residual traffic
• C+ analyzer (order of sec)
• Separation is more efficient (less accurate)
using multi-scale analysis
Rice University, SPiN Group
spin.rice.edu
Beta Traffic Component
•
•
•
•
Constitutes main load
Governs LRD properties of overall traffic
Is Gaussian at sufficient utilization (Kurtosis = 3)
Is well matched by ON/OFF model
400
350
Traffic percentile: 1%
Beta Traffic99%
Kurtosis 3.4
beta =
Mean
300
99%
250
200
150
Mean
Recall:
non-dominating
100
50
0
0
0.5
1
1.5
2
2.5
x 10
Beta traffic
Rice University, SPiN Group
5
Histogram
of
Beta
traffic
Variance
time
plot
Number
of
connections
=
ON/OFF
spin.rice.edu
Origins of Alpha Traffic
• Mostly TCP (flow & congestion controlled).
• HTTP, SMTP, NNTP, etc.
• Alpha connections tend to have the same sourcedestination IP addresses.
• Any large volume connection with the same e2e IP
addresses as an alpha connection is also alpha.
 Systematic cause of alpha:
Large file transfers over high bandwidth e2e paths.
Rice University, SPiN Group
spin.rice.edu
Simple Connection Taxonomy
Bursts arise from large
transfers over fast links.
This is the only systematic reason
cwnd
bandwidth
.
RTT
Rice University, SPiN Group
spin.rice.edu
Cwnd or RTT?
Colorado State University trace, 300,000 packets
1/RTT (1/s)
10
10
10
Beta
10
Alpha
1
cwnd (B)
10
2
0
10
10
5
Beta
Alpha
4
3
peak rate
 cwnd
1/RTT
2
-1
10
3
10
4
10
peak-rate (Bps)
5
10
Correlation coefficient=0.68
6
10 3
10
10
4
10
5
peak-rate (Bps)
10
Correlation coefficient=0.01
RTT has strong influence on bandwidth and dominance.
Rice University, SPiN Group
spin.rice.edu
6
Examples of Alpha/Beta Connections
one alpha connection
(96078, 196, 80, 59486)
one beta connection
1400
1400
1200
1200
packet size
size (bytes)
(bytes)
packet
1000
1000
800
800
600
600
forward direction
forward direction
400
400
200
200
0
0
reverse direction
-200
-200
0.4
0.6
41.3
0.8
1
reverse direction
1.2
41.35
1.4
1.6
1.8
41.4
2
2.2
2.4
41.45
packet arrival
arrivaltime
time(second)
(second)
packet
Alpha connections burst because of
short round trip time, not large TCP window
Rice University, SPiN Group
spin.rice.edu
Modeling of Alpha Traffic
• ON/OFF model revisited:
High variability in connection rates (RTTs)
Low rate = beta
+
High rate = alpha
+
+
=
=
fractional Gaussian noise
stable Levy noise
Modeling of Alpha Traffic
• ON/OFF model revisited:
High variability in connection rates (RTTs)
Low rate = beta
x 10
High rate = alpha
5
x 10
5
3
2
number of bytes
number of bytes
2.5
1.5
1
0.5
0
0
2
1.5
1
0.5
2000
4000
time (1 unit=500ms)
6000
fractional Gaussian noise
0
0
2000
4000
time (1 unit=500ms)
6000
stable Levy noise
Rice University, SPiN Group
spin.rice.edu
Impact: Simulation
• Simulation: ns topology to include alpha links
Simple: equal bandwidth
Realistic: heterogeneous
end-to-end bandwidth
• Congestion control
• Design and management
Rice University, SPiN Group
spin.rice.edu
Impact: Performance
• Beta Traffic rules the small Queues
• Alpha Traffic causes the large Queue-sizes
(despite small Window Size)
All Alpha Packets
Rice University, SPiN Group
Queue-size with Alpha Peaks
spin.rice.edu
Multiscale Nature of Traffic
• Multifractal
– Small time scale
– Mixture Gaussian - Stable
– Network topology
– Control at Connection level
• LRD
– Large time scales
– approx. Gaussian
– Client behavior
– Bandwidth over Buffer
packet
scheduling
round-trip
time
session
lifetime
< 1 msec
msec-sec
minutes
Structure: Multiplicative
Rice University, SPiN Group
network
management
hours
Additive
spin.rice.edu
Ongoing Work
• Versatile efficient additive/multiplicative
models (fast synthesis, meaning of model parameters)
• Impact of Alpha traffic on Performance
• Goal: Obtain performance parameters
directly from network and user specifications
• Monitoring tool for network load (freeware)
using chirps (Caida, SLAC)
• Verification/”internal” measurements using
IP-tunneling
Rice University, SPiN Group
spin.rice.edu
Summary
• Alpha traffic (peak rate > burst threshold)
– Few connections. Responsible for bursts
– Origin: Large transfer over high bandwidth paths
– High bandwidth from short RTT
• Beta traffic (residual):
– Main load. Responsible for LRD
– Origin: Crowd with limited bandwidth
– Gaussian at sufficient utilization
• Connection level analysis: parameter estimation
drastically simplifies using multi-scale analysis
• Statistical modeling: requires novel multi-scale
models (mixtures of additive and multiplicative trees)
Rice University, SPiN Group
spin.rice.edu