Download EarlyWarning

Monitoring and Early Warning
for Internet Worms
Cliff C. Zou, Lixin Gao, Weibo Gong, Don Towsley
Univ. Massachusetts, Amherst
1
How to detect an unknown
worm at its early stage?

Monitoring:

Monitor worm scan traffic (non-legitimate traffic).



Observation data is very noisy.



Connections to nonexistent IP addresses.
Connections to unused ports.
Old worms’ scans.
Port scans by hacking toolkits.
Detecting:


Anomaly detection for unknown worms
Traditional anomaly detection: threshold-based


Check traffic burst (short-term or long-term).
Difficulties: False alarms; threshold tuning.
2
“Trend Detection”
 Detect traffic trend, not burst
Trend: worm exponential growth trend at the beginning
Detection: the exponential rate should be a positive, constant value
Monitored illegitimate traffic rate
60
60
60
50
50
50
40
40
40
30
30
20
20
10
10
0
0
10
20
30
40
50
30
20
10
0
10
20
30
40
50
Exponential rate a on-line estimation
0.2
0.2
0.2
0.15
0.15
0.1
0.1
0.1
0.05
0.05
0.05
0
0
0
-0.05
-0.05
-0.1
-0.1
0.15
10
20
30
40
50
10
20
10
20
30
40
50
-0.05
-0.1
10
Non-worm traffic burst
20
30
40
50
30
Worm traffic
40
50
3
Why exponential growth
at the beginning?


The law of natural growth  reproduction
Exponential growth — fastest growth pattern when:




Negligible interference (beginning phase).
All objects have similar reproductive capability.
Large-scale system — law of large number.
Fast worm has exponential growth pattern



Attacker’s incentive: infect as many as possible before
counteractions.
If not, a worm does not reach its spreading speed limit.
Slow spreading worms can be detected by other ways.
4
Worm modeling —
simple epidemic model
# of contacts  I  S
Simple epidemic model:
It
# of infected hosts
: # of susceptible
: # of infectious
: Total # of hosts
: Infectious ability
100
200
300
400
500
600
Time
10
# of infected hosts
Discrete model:
10
with exponential rate
At very early stage:
10
4
2%
1%
3
2
50
100
150
Time t
200
250
5
Why use simple epidemic model?


Can model most scan-based worms.
We can use other worm models as well with minor
modifications (such as exponential model).
Code Red
SQL Slammer
Figures from:
D. Moore, V. Paxson, S. Savage, C. Shannon, S. Staniford, N. Weaver,
“Inside the Slammer Worm”, IEEE Security & Privacy, July 2003.
6
Kalman Filter Estimation

Equivalent to Recursive Least Square Estimator:

Give estimation at each discrete time.
 Robust to noise.
System: Discrete-time simple epidemic model

System state:




Worm infection rate a. (a= bN, exponential growth rate at beginning)
Epidemic parameter b. (worm infectious ability)
Measurement from monitors:

Ci : cumulative # of observed infected, Zi :# of scans at time i.
7
Kalman Filter Estimation
System:
where
Kalman Filter for estimation of Xt :
8
Code Red simulation
experiments
Population: N=360,000,
Infection rate: a = 1.8/hour,
Scan rate h = N(358/min, 1002), Initially infected: I0=10
Monitored IP space 220,
Monitoring interval: D = 1 minute
Consider background noise
x 10
5
3.5
Real value of a
Estimated value of a
0.25
Estimated infection rate
# of infected hosts
3
Infected I t
Observed Infected Ct
2.5
2
0.2
0.15
1.5
1
0.05
0.5
0
0
0.1
100
200
300
400
Time t (minute)
500
600
0
150
200
250
Time t (minute)
300
350
Before 2% (223 min): estimate is already stabilized and
oscillating a little around a positive constant value
9
SQL Slammer simulation
experiments
Population: N=100,000,
Monitored IP space 220,
Scan rate h = N(4000/sec, 20002),
Initially infected: I0=10
Monitoring interval: D = 1 second, Consider background noise
10
x 10
4
0.5
Estimated infection rate
# of infected hosts
# of infected hosts It
8
6
4
2
0
0
50
100
Time t (second)
150
200
Real value of a
Estimated value of a
0.4
0.3
0.2
0.1
0
20
40
60
Time t (second)
80
100
Before 1% (45 sec): estimate is already stabilized and
oscillating around a positive constant value
10
Early detection of Blaster
3.5

Blaster: sequentially scans from a
starting IP address:


3
1.5
1
0.5
0
0
5
100
200
300 400 500
Time t (minute)
600
After using low-pass filter
3.5
4
3
# of infected hosts
1024-block monitoring
16-block monitoring
2
It follows simple epidemic model.
x 10
4
2.5
40% from local Class C address.
60% from a random IP address.

x 10
x 10
3.5
2.5
4
1024-blocks monitoring
16-block monitoring
3
2
2.5
2
1.5
1.5
1
0.5
0
100
200
95 percentile (Blaster)
5 percentile (Blaster)
95 percentile (Code Red)
5 percentile (Code Red)
300 400 500 600
700
Time t
1
0.5
0
0
100
200 300 400
Time t (minute)
500
600
700
11
700
Bias correction for
uniform-scan worms

Bernoulli trial for a worm to hit monitors (hitting prob. = p ).
Bias correction:
: Average scan rate
3.5
# of infected hosts
3
2.5
5
4
Infected hosts It
Observed infected Ct
Estimated It after
bias correction
3.5
# of infected hosts
x 10
2
1.5
1
2.5
5
Infected hosts It
Observed infected Ct
Estimated It after
bias correction
2
1.5
1
0.5
0
3
x 10
0.5
100
200
300 400 500
Time t (minute)
600
Monitoring 217 IP space
700
0
100
200
300
400 500
Time t (minute)
600
Monitoring 214 IP space
Bias correction can provide unbiased estimate of It
12
Prediction of
Vulnerable population size N
x 10
5
3.5
Direct from Kalman filter:
3
2.5
2
1.5

1
0.5
0
h : A worm sends out h scans per D time
(derived from egress scan monitor)

Estimated population N
Alternative method:
6
x 10
5
100
200
300
400
500
600
5
4
3
2
1
0
50
Real population size N
From estimated b
From h and estimated a
100
150
200 250 300
Time t (minute)
350
400
Estimation of population N
13
Use exponential growth model
4
2%
1%
# of infected hosts
# of infected hosts
10
At the early stage:
10

10

3
2
100
50
: observation data
Model #2: Autoregressive (AR) model
200100 300
500
150400 200
Time
Timet
600
250
Early stage of
worm propagation

Model #3: Transformed linear model

14
Comparison between
three estimation models
0.2
0.2
0.2
0.15
0.15
0.15
0.1
0.1
0.1
0.05
0.05
0.05
0
0
140
160
180
200
Time t (minute)
220
240
Epidemic model

0
140
160
180
200
Time t (minute)
220
240
AR exponential model
140
160
180
200
Time t (minute)
220
240
Transformed linear model
Observations



AR exponential model is smoother than epidemic model
Transformed linear model gives best results
Detect a worm when it infects about 0.5% population
15
Simple analysis of
three estimation models

Why AR exponential model is smoother than epidemic
model?


Introduced errors from measurement data:
 Epidemic model
 AR exponential model
Why transformed linear model is better than AR
model?
assume
AR exponential model:
Transformed linear model:
where
16
Summary

Trend detection: non-threshold-based methodology






Principle: detect traffic trend, not burst
Pros : Robust to background noise  low false alarm rate
Cons: Rely on worm model, representation of measurement data
Epidemic model, exponential model
Using low-pass filter on noisy observation data
For uniform-scan worms

Bias correction:

Forecasting N:
( IPv4 )

: scanning IP space
: scan hitting prob.
 Routing worm
: Average scan rate
: Infection rate
: cumulative # of observed infectious
17