Download Denial of Service Attack and Prevention System

Denial of Service Attack and Prevention System
Design Document
Andrew Barkley
Matt Dingfield
Quoc Thong Le Gia
Yashodhan Gokhale
Sunday, May 07, 2000
Denial of Service Attack and Prevention System ............................................................... 1
Attack Daemons .................................................................................................................. 2
Attack Daemon Protocol ................................................................................................. 2
Attack Daemon Process .................................................................................................. 2
DoS Server .......................................................................................................................... 3
Log Server Protocol ........................................................................................................ 3
Router Control Daemon ...................................................................................................... 4
Router Topology Configuration ...................................................................................... 4
upstream ...................................................................................................................... 4
downstream ................................................................................................................. 5
Router Control Protocol .................................................................................................. 6
Router Control Algorithm ............................................................................................... 7
The Algorithm ............................................................................................................. 7
Implementation Interfaces .......................................................................................... 7
Router Control History ............................................................................................... 8
Denial of Service Attack and Prevention System
The Denial of Service (DoS) Attack and Prevention System is a suite of applications and protocols that
work together to provide a centrally controllable platform for generating denial of service attacks, detecting
them, and blocking their malicious effects.
At the core of the system are the routers themselves. The router machines are Linux x86 machines with
multiple network interfaces. They are connected with a tree topology, and programmed with static routes.
Software, known to us as the Router Control daemon, runs on each of these machines. This software is
configured with the location of its neighbors, so that it may know which routers it may communicate with
using the Router Control Protocol. The protocol allows for configuration of the traffic blocking behavior
of the local router, and retrieval of information about the performance of the local router and its children.
The denial of service attacks under study with the prototype are those dealing with authentic HTTP traffic.
Therefore, the measurements of traffic that the router makes count only such traffic, and the filters can
block only that traffic. Therefore, the only hindrances to non-HTTP traffic are a loss of bandwidth due to
the attack and possibly a performance hit on the routers running the Router Control process.
The HTTP traffic is generated by Attack daemons that may run on any machine in the test network,
independent of the Router Control software. They use a simple ASCII-stream protocol for configuration
and launch of attacks. These processes do not coordinate with each other, as do the Router Control
processes, but rather rely on another process to trigger them concurrently.
The Router Control software would be inefficient, and perhaps impossible, to run for all protocols when
considering scenarios beyond HTTP attacks. Therefore, its role is merely prevention of a known attack.
The detection of attacks is left to server software that is being exploited by the denial of service attack. As
we are focusing on HTTP attacks, the open-source web server Apache is used as the detection component.
A modified Apache web server is run on the route node of the hierarchy, which also runs the Router
Control daemon and a yet to be discussed process that serves many centralizing functions in the system,
known as the DoS Server. The Apache web server has been modified to send information to other
processes on the local machine upon each new connection, namely the DoS Server. The DoS Server logs
this data, implements a protocol that allows for retrieval of the data, and implements a protocol that allows
for configuration of a threshold rate at which the DoS Server will automatically enable the Router Control
DoS prevention software for HTTP. Therefore, the DoS Server is the detection mechanism of the system.
The DoS Server also wears another hat as a proxy through which to coordinate attacks with the Attack
daemons, so that experiments may be performed by a user interface off of the isolated network, as long as
the root node of the network hierarchy is accessible. This proxy is necessary to run attacks from any
machine off of the test network due to avoid issues with configuring more static routes.
The DoS Server is the right-hand process of the centralized experiment control software, called DoS GUI.
The DoS GUI. Provides interaction with the other processes using facilities provided by the DoS server. It
provides a visualization and storage for performance data retrieved from both the DoS Server and the
Router Control processes, as well as the ability to launch HTTP DoS attacks.
1
Attack Daemons
The Attack daemons are very simple but extremely resource consuming processes that run on each
designated attack machine. The process provides a protocol listener on a known that accepts commands for
configuration and launch of HTTP attacks. An attack consists of many “threads,” or lightweight processes,
that each download documents from the target web server. There is one “smart thread” that traverses all
links to content local to the web server, and generates a large list of target URLs. The other threads simply
choose a URL out of that list and download it. It is a very simple way to generate traffic on the web server.
The Attack daemon can be configured to have few threads so that it seems like normal low-load traffic, as
the few threads are the only interacting processes, so there will be no more than that many web transactions
at a given instant. When multiple daemons are polling the web server with thousands of threads, the rate of
requests exceed the rate at which the server can service them, and the processes are responsible for a DoS
attack.
Attack Daemon Protocol
The protocol is a simple one-way flow of information command set over an ASCII stream. The standard
TCP server port for the service is 2001. The following lines following the “” are commands. All
commands are case insensitive
 attack
Start the attack as configured
 site
The base URL to start the attack from
 threads
The number of clients to attack with
 duration
The duration in seconds for the attack to last
 quit
Exit the command session
Attack Daemon Process
The Attack daemon may be loaded with the following command:
java –classpath attack.jar Attack
The command simply loads the correct Java class from the given archive.
2
DoS Server
The DoS Server implements the interaction with Apache, logging and availability of Apache data, detection
and response to HTTP attacks, and an Attack daemon proxy. It is a less hectic design than it first seems.
There is a thread names UDP Counter that listens on UDP port 4950 for packets from Apache. When it
receives a packet, it increments an internal counter. Another thread named Packet Logger divides the
counts up into intervals by polling the UDP Counter. The thread Log Server is the ASCII command server
that listens on TCP port 8567 for all of the previously described commands. A data class named Packet
Log stores all of the log data created by the Packet Logger and served by the Log Server. Beyond storing
the data, it also triggers the Router Control daemon to enable prevent mode when the threshold rate is
reached.
Log Server Protocol
The following is the ASCII stream protocol implemented in the Log Server. Lines starting with “” are
sent by the client, and those starting with “” are replies from the server. Exactly one acknowledgement
line is sent for each input command. All acknowledgements begin with either “OK” or “FAIL” to indicate
completion status. All commands and replies are case insensitive.
 quit
 ok
This command quits the Log Server session.
 logs
 ok <logs>
This command retrieves the log entries for all recent intervals, up to the maximum storable on the machine
(set at 500). Here, <logs> is the series of log records. Each record is separated with a “%” character to aid
in parsing. A log itself is represented as five string values separated with the “|” character. The string
values are (in order) time, packet count, duration of log interval in seconds, threshold for triggering Router
Control, and whether or not the interval was over threshold or not.
 duration <duration>
 ok <duration>
This command retrieves and optionally sets the current duration of the log interval in seconds for
successive logs generated with the Packet Logger. Here, <threshold> is an integer.
 threshold <threshold>
 ok <threshold>
This command retrieves and optionally sets the current threshold of packets per window received by the
UDP Counter before the Router Control is enabled with a “prevent on” signal.
 attack <settings>
 ok
or
 fail <information>
This command triggers the Attack daemon proxy to relay an attack to the host and parameters in <settings>.
Here, <settings> is a space separated list of options. The options are <site> (URL), <threads> (integer),
and <duration> (integer), respectively. The command returns a failure when not able to trigger the attack.
3
Router Control Daemon
The Router Control daemon runs on each of the downstream routers and the server. It is ultimately
responsible for preventing hot packets from reaching the server during a denial of service attack. The
Router Control daemon running on the server is the root of a tree of such daemons. The root processes will
propagate commands down toward the leaves of the tree as appropriate.
The Router Control daemon may be loaded with the following command:
java –classpath router.jar Ragent
Router Topology Configuration
The Router Control daemon does not have the ability to discover its neighboring daemons automatically.
The router.cfg file provides topology information to the Router Control daemon. The format of this file is
as follows:
<neighbor count>
<self> <local name> <local ip>
upstream <upstream name> <upstream ip> <port>
downstream <downstream name> <downstream ip> <port>
…
<neighbor count>
An integer giving the total number of adjacent Router Control daemons.
<self>
This tells the daemon that it is the root of the control tree. On the root machine, <self> is
server. Otherwise, <self> is host.
<local name>
This is the name of the local machine ie. pecan.cs.tamu.edu.
<local ip>
This is the IP address of the local machine in dotted decimal notation ie. 128.194.130.56.
upstream
The line beginning with upstream gives the location of the parent daemon of the local
daemon. If this is the root, then the local machine is given. This odd notation allows the
DoS Server (which runs on the same machine as the root daemon) to be recognized as a
legitimate process to send commands.
<upstream name>
4
This is the name of the machine running the parent of the daemon. If the daemon is the root
daemon, then this is the name of the local machine and is equal to <local name>.
<upstream ip>
This is the IP address of the machine running the parent daemon. If the daemon is the root
daemon, then this is the IP address of the local machine and is equal to <local ip>.
<port>
This is the TCP port number to expect communications from the parent daemon. If this is the
root, then it will expect communications from the DoS Server on this port.
downstream
The line beginning with downstream gives a child daemon of the local daemon. There can be multiple
downstream lines, one for each child. If the local daemon is a leaf, then there will be no downstream lines.
<downstream name>
This is the name of a machine running a child daemon of the local daemon.
<downstream ip>
This is the IP address of a machine running a child daemon of the local daemon.
<port>
This is the TCP port number the local daemon should use when communicating with this child
daemon.
5
Router Control Protocol
Communication between the DoS Server and the root Router Control daemon, and between a daemon and
each of its children is accomplished with the Router Control Protocol. This is a master/slave protocol
where the master issues a command and the slave carries out the command and returns a response when
complete. The communication is carried by an ASCII character stream over TCP on a port specified in the
router topology configuration (router.cfg). In communications between DoS Server and the Router Control
daemon, the DoS Server acts as the master and the root daemon acts as the slave. In communications
among the Router Control daemons, a parent acts as a master and the child acts as the slave. Note that
there is no communication among sibling daemons. Finally, a Router Control daemon can handle multiple
simultaneous requests from multiple masters. This allows operations to be carried out in parallel.
 lines come from the server
 lines come from the router
The following transactions may occur (case insensitive):
Turning on prevention should only fail if already on.
 prevent on
 OK <additional info>
or
 FAIL <additional info>
Turning off prevention should fail if there exists a downstream router that is not responding.
 prevent off
 OK <additional info>
or
 FAIL <additional info>
You can set the threshold manually, and it will only fail for an invalid number. (It may be appropriate to
extend this to fail if a downstream router cannot be reached.)
 threshold <n>
 OK <additional info>
or
 FAIL <additional info>
You can get the current threshold value with (never fails):
 threshold
 OK <threshold value>
You can set the duration (seconds) manually, and it will only fail for an invalid number.
 duration <n>
 OK <duration info>
or
 FAIL <additional info>
You can get the current duration value with (never fails):
 duration
 OK <duration value>
You can retrieve status information for a router's subtree of routers with the following:
 status
 OK <status info>
6
Where <status info> is a sentence containing space-separated information for each downstream router, that
again recurse on their downstream routers.
(ip <status>… (-recurse-)…)…
<status> is a word that can be repeated many times, and looks like the following (word is actually enclosed
in angle brackets, placeholder values are not):
<off:<time>>
or
<on:<blocking>:<time>:<packets>:<window>>
values:
<blocking>:
<time>:
<packets>:
<window>:
true or false
Standard Java Date.toString() format
Positive integer
Positive integer of duration in seconds for counting packets starting at
<time>
Router Control Algorithm
The Router Control daemon is one piece of a distributed system for controlling traffic and preventing
denial of service attacks. Described here is the algorithm employed internally in the Router Control
daemon to control local traffic.
The Algorithm
When activated, the Router Control daemon acts as a threshold gate for HTTP packets destined for the
server. The gate is controlled by two parameters. The threshold is measured in packets per second, and
the shutoff duration is measured in seconds. Both of these parameters are configurable via the Router
Control Protocol. Internally, the daemon has two other parameters. The packet window is the time
interval over which packets are counted. The resolution is the refresh period of the packet rate calculation.
1.
2.
3.
4.
5.
6.
7.
Wait for resolution time
Get number of packets received since last check
Add this number to next entry in circular queue (drops oldest entry in circular queue)
Sum entries of circular queue and divide by packet window to get packet rate
If the packet rate > threshold then reset timer and block packets if not already blocked
Otherwise, if we are blocking packets and the timer has timed out then unblock packets and clear timer
If we haven’t been told to stop then go back to step 1
In step 2, the daemon only counts HTTP (TCP port 80) packets sent from the router toward the server to
compute the packet rate in step 4.
Implementation Interfaces
Both the packet counting in step 2 and the blocking of packets in step 5 are implemented using the Linux
firewall interface. The firewall interface is a facility in Linux to aid in writing firewalls, among other
things. A complete description of this interface is available by viewing the man page for ipfw. (ipfw is
automatically available on Red Hat Linux 6.1 installation. Its availability on other distributions is
unknown.) The firewall interface uses chains and rules. A chain is basically a sequence of rules and a rule
7
is a description of a packet and some action to take when a packet is found that matches the description. In
the Router Control daemon, the description is that of a TCP packet on port 80 with destination address
equal to the address of the server machine. There are always at least three chains available: input, output,
and forward. Input is applied to packets received on a network interface, output is applied to packets sent
on a network interface, and forward is applied to packets being forwarded, but not generated, by the local
machine.
When the threshold gate of the Router Control becomes active, it removes all rules from the three chains
mentioned and installs one rule in the output chain that matches the packets it is interested in. This rule
contains no action and so does not affect packets that match. It does have a very useful side effect.
Whenever Linux finds that a packet matches a rule, it increments the 64 bit packet and byte counters. By
reading the value of the packet counter we can know how many packets the machine has sent.
The counter value is read by parsing the file /proc/net/ip_fwchains. This is not a true file, but a pipe to the
Linux kernel that always contains current information on all installed chains.
To block packets, a second rule is installed that immediately follows the rule used for packet counting.
This way, the new rule does not interfere with the packet counting function of the first rule. The second
rule is identical to the first except it contains the action “DENY” which informs Linux to drop all packets
matching the rule. When a Linux drops a packet due to the “DENY” action, the packet disappears and no
error messages are generated.
There are two important consequences of the method used to implement packet counting and packet
blocking. First, because the code relies on the absolute position of the rules within the ip_fwchains file, it
will not function in the presence of another process that manipulates firewall chains. This could be
overcome by a more sophisticated use of chains. Second, a process must have root access to manipulate
chains and read the ip_fwchains file.
Router Control History
Prior to using the firewall interface for packet counting and packet blocking, we tried a different approach
to packet counting and packet blocking. The Router Control used packet capturing to count packets and it
installed a rejecting route into the routing table to block packets.
Packet capturing is the technique used by tcpdump to get a copy of packets entering the kernel. Tcpdump
uses a relatively high level packet capture library developed by the same people as tcpdump. The previous
approach to counting packets used a modified version of tcpdump that ran for some specified length of time
before returning with the number of packets received during the same period. We did not use the packet
capture library directly because it is not well documented.
Installing a rejecting route into the routing table causes the kernel to consider that route unreachable. Any
packets received by the kernel that match a rejecting router are dropped and an ICMP “Router
Unreachable” message is sent back to the source.
We abandoned these methods for several reasons. The Router Control daemon was highly unstable. It
would consistently experience bizarre failures. In one instance, the creation of a socket in Java would
consistently freeze the thread. Other times it would experience segmentation faults. We never determined
the exact cause of these failures. We hypothesized several possible explanations.
1.
2.
3.
The particular version of tcpdump used was the latest alpha release. Perhaps it contained
subtle memory errors that were corrupting the systems memory.
There could be unknown side-effects to running the network adapter in promiscuous mode.
It pushed the JNI to hard. Perhaps there are deficiencies in the JNI.
8
The use of rejecting routes was abandoned because the generation of ICMP messages generates additional
traffic on an already loaded network link. The new method creates a temporary black hole so no additional
overhead is created. It also seems more appropriate because the route is not truly unreachable, but rather
overloaded.
9