Download CS244a: An Introduction to Computer Networks

Brief Overview of
Networking Evaluation
Methods and Tools
winter 2008
Evaluation Tools
1
Outline
• Tools
– Simulation
• ns2
– Cluster based network emulation
• Emulab
– Live Distributed Testbed
• PlanetLab
• Questions
– What is the right tool(s) for my research?
– What does it take to get up and running?
winter 2008
Evaluation Tools
2
Outline
• For each tool/testbed
– How does it work?
– A “hello world” in ___
– What kind of research is it good for?
• Tradeoffs between tools
winter 2008
Evaluation Tools
3
Simulation
(ns2)
winter 2008
Evaluation Tools
4
What is ns?
• Network simulator
• a discrete event simulator
• focused on modeling network protocols
–
–
–
–
–
wired, wireless, satellite
TCP, UDP, multicast, unicast
Web, telnet, ftp
Ad hoc routing; sensor networks
Infrastructure: stats, tracing, error models etc.
winter 2008
Evaluation Tools
5
ns --- what is it good for?
Used to:
•
•
•
Evaluate performance of existing
network protocols.
Prototyping and evaluation of new protocols.
Large-scale simulations not possible
in real experiments.
winter 2008
Evaluation Tools
6
ns
How does it work:
• Event-driven simulator
–
–
–
–
Model world as events
Simulator has list of events
Process: take next one, run it, until done
Each event happens in instant of virtual time, but takes
arbitrary real time
• Single thread of control
• Packet level
winter 2008
Evaluation Tools
7
Ns models
• Traffic/applications
– CBR, FTP, telnet, web
• Routing/Queuing
– Drop-tail, FQ, SFQ, RED, DRR
– Wired routing, adhoc routing etc
• Transport
– TCP (variants), UDP, multicast (SRM)
winter 2008
Evaluation Tools
8
ns - software structure
• Object oriented (C++, OTcl) – code reuse
• Scalability + Extensibility
– Control/”data” separation
– Split C++/OTcl object
• C++ for packet-processing (fast to run)
• OTcl for control - (fast to write)
– Simulation setup and configuration
winter 2008
Evaluation Tools
9
otcl and C++: The Duality
Pure C++
objects
Pure OTcl
objects
Your ns-script
OTcl
C++/OTcl split
objects
C++
ns
winter 2008
Evaluation Tools
10
Outline
•
•
•
•
•
Overview
Tcl, OTcl basics
ns basics
Extending ns
ns internals
winter 2008
Evaluation Tools
11
Basic structure of ns-scripts
•
•
•
•
•
•
Creating the event scheduler
[Tracing]
Creating network topology
Creating Transport Layer - Agents
Creating Applications - Applications
Events!
winter 2008
Evaluation Tools
12
Creating Event Scheduler
• Create scheduler
– set ns [new Simulator]
• Schedule event
– $ns at <time> <event>
– <event>: any legitimate ns/tcl commands
• Start scheduler
– $ns run
winter 2008
Evaluation Tools
13
“Hello World” in ns
simple.tcl
set ns [new Simulator]
$ns at 1 “puts \“Hello World!\””
$ns at 1.5 “exit”
$ns run
bovik@gs19% ns simple.tcl
Hello World!
bovik@gs19%
winter 2008
Evaluation Tools
14
• Nodes
Creating Network
– set n0 [$ns node]
– set n1 [$ns node]
• Links & Queuing
– $ns duplex-link $n0 $n1 <bandwidth> <delay>
<queue_type>
– Queue type: DropTail, RED, CBQ, FQ, SFQ, DRR
winter 2008
Evaluation Tools
15
• Unicast
Routing + traffic
– $ns rtproto <type>
– <type>: Static, Session, DV
• Multicast support also.
• Traffic
– Simple two layers: transport and application.
– Transport: TCP, UDP etc.
– Applications: web, ftp, telnet etc.
winter 2008
Evaluation Tools
16
• UDP
The transport layer: UDP
– set udp [new Agent/UDP]
– set null [new Agent/NULL]
– $ns attach-agent $n0 $udp
– $ns attach-agent $n1 $null
– $ns connect $udp $null
winter 2008
Evaluation Tools
17
The transport layer: TCP
• TCP
– set tcp [new Agent/TCP]
– set tcpsink [new Agent/TCPSink]
– $ns attach-agent $n0 $tcp
– $ns attach-agent $n1 $tcpsink
– $ns connect $tcp $tcpsink
winter 2008
Evaluation Tools
18
FTP
Creating Traffic: On Top of
TCP
– set ftp [new Application/FTP]
– $ftp attach-agent $tcp
– $ns at <time> “$ftp start”
Telnet
– set telnet [new Application/Telnet]
– $telnet attach-agent $tcp
winter 2008
Evaluation Tools
19
Creating Traffic: On Top of
UDP
• CBR
– set src [new Application/Traffic/CBR]
• Exponential or Pareto on-off
– set src [new Application/Traffic/Exponential]
– set src [new Application/Traffic/Pareto]
• Trace driven traffic
– Inter-packet time and packet-size
winter 2008
Evaluation Tools
20
Attaching a traffic source
• set cbr [new Application/Traffic/CBR]
• $cbr attach-agent $udp
• $ns at <time> “$cbr start”
winter 2008
Evaluation Tools
21
Tracing
Trace packets on all links:
– set f[open out.tr w]
– $ns trace-all $f
– $ns flush-trace
– close $f
<event><time><from><to><type><size>--<flags>--<flow id><src><dst><seqno> <pckt
id>
+ 1 0 2 cbr 210 ------- 0 0.0 3.1 0 0
- 1 0 2 cbr 210 ------- 0 0.0 3.1 0 0
r 1.00234 0 2 cbr 210 ------- 0 0.0 3.1 0 0
Is tracing all links always the best thing to do?
winter 2008
Evaluation Tools
22
More Tracing
• Tracing specific links
– $ns trace-queue $n0 $n1 $f
• Tracing variables
– set cwnd_chan_ [open all.cwnd w]
– $tcp trace cwnd_
– $tcp attach $cwnd_chan_
winter 2008
Evaluation Tools
23
Controlling object parameters
• Almost all ns objects have parameters
– ex. Application/Traffic/Exponential has rate and packetSize
– set parameters in OTcl
• set etraf [new Application/Traffic/Exponential]
• $etraf set rate_ 1Mb
• $etraf set packetSize_ 1024
winter 2008
Evaluation Tools
24
Putting it all together
set ns [new Simulator]
set n0 [$ns node]
set n1 [$ns node]
$ns duplex-link $n0 $n1 1.5Mb 10ms
DropTail
n0
1.5Mb 10ms
FTP/TCP
n1
Creating Topology
$ns trace-queue $n0 $n1 $f
set tcp [$ns create-connection TCP
$n0 TCPSink $n1 0]
Creating Transport layer
set ftp [new Application/FTP]
$ftp attach-agent $tcp
Creating Applications
$ns at 0.2 "$ftp start“
Schedule Events
$ns at 1.2 ”exit“
winter 2008
$ns run
Evaluation Tools
25
Example: XCP vs CSFQ
• Compare
utilization of links
• Compare
throughput of
flows
winter 2008
Evaluation Tools
26
nam – the network animator
set nf [open out.nam w]
$ns namtrace-all $nf
…
exec nam out.nam &
n0
winter 2008
n1
Evaluation Tools
27
ns “components”
• ns, the simulator itself
• nam, the Network AniMator
– Visualize ns output
– GUI input simple ns scenarios
• Pre-processing:
– Traffic and topology generators
• Post-processing:
– Simple trace analysis, often in Awk, Perl, or Tcl
winter 2008
Evaluation Tools
28
Network Dynamics: Link
failures
• $ns rtmodel-at <time> <up|down> $n0 $n1
• $ns rtmodel Trace <config_file> $n0 $n1
• $ns rtmodel <model> <params> $n0 $n1
<model>: Deterministic, Exponential
winter 2008
Evaluation Tools
29
Issues in Simulations
• Suppose you want to study the way TCP sources
Agent/FTP
share a bottleneck link…
Agent/FTP
Which topology?
Agent/FTP
Agent/FTP
Which traffic sources?
Background Traffic?
When to start sources?
What else affects results?
winter 2008
Evaluation Tools
30
Cluster Based Emulation
(Emulab)
368 nodes
3 big Ciscos
Slides based on SOSP poster by Jay Lepreau et. al.
University of Utah www.emulab.net
winter 2008
Evaluation Tools
31
Why?
•
•
•
•
•
•
•
•
•
•
“We evaluated our system on five nodes.”
-job talk from university with 300-node cluster
“We evaluated our Web proxy design with 10 clients on 100Mbit ethernet.”
“Simulation results indicate ...”
“Memory and CPU demands on the individual nodes were not measured, but
we believe will be modest.”
“The authors ignore interrupt handling overhead in their evaluation, which
likely dominates all other costs.”
“You have to know the right people to use the cluster.”
“The cluster is hard to use.”
“<Experimental network X> runs FreeBSD 2.2.x.”
“February’s schedule for <Testbed Y> is…”
“<Network Z> is tunneled through the Internet”
winter 2008
Evaluation Tools
32
What?
• An instrument for experimental CS research
• A completely configurable “Internet emulator” in a room
– At its core, it’s bare hardware, with…
– … complete remote access and control
• But, also simple to use
– Lots of fast tools for common case
• Automatic topology, node and link configuration
• Automatic traffic generation
– Universally available
• Universities, research labs, companies
– Zero-penalty for remote research
winter 2008
Evaluation Tools
33
Key Design Aspects
• Allow experimenter complete control
– Configurable network properties
• link bandwidth, Latency, and loss rates via transparently
interposed “traffic shaping” nodes that provide WAN
emulation
– Configurable OS image
• Linux, FreeBSD, Windows
• Virtualization
– of all experimenter-visible resources
– node names, network interface names, network addrs
• e.g., node-0.esmexp1.esm.emulab.net
winter 2008
Evaluation Tools
34
Emulab Architecture
Internet
Web/DB/SNMP
Switch Mgmt
Users
PowerCntl
Control Switch/Router
Serial
PC
PC
Sharks
16
8
Sharks
160
Programmable “Patch Panel”
winter 2008
Evaluation Tools
35
Experiment Creation Process
winter 2008
Evaluation Tools
36
Using Emulab
• Submit ns script via web form
– Specify number of nodes
– Specify link properties
• Relax while emulab …
–
–
–
–
–
–
Generates config from script & stores in DB
Maps specified virtual topology to physical nodes
Provides user accounts for node access
Assigns IP addresses and host names
Configures VLANs
Loads disks, reboots nodes, configures OSs
• Two ways to run experiments
– Interactive
• Works only if there are enough free nodes right now
– Batch
• Runs experiment when enough free nodes available
winter 2008
Evaluation Tools
37
Using Emulab
• Time sharing model for nodes
– If you check out 50 nodes, they are exclusively
yours until you release them (or forced to
release them!)
winter 2008
Evaluation Tools
38
Using Emulab
set ns [new Simulator]
source tb_compat.tcl
##setup the core nodes
for {set i 0} {$i <= 4} {incr i} {
set node($i) [$ns node]
tb-set-node-os $node($i) FC4-STD
}
## setup the core links
for {set i 0} {$i < 4} {incr i} {
set j [expr $i + 1]
$ns duplex-link $node($i) $node($j) 100Mb 0ms DropTail
}
$ns duplex-link $node(4) $node(0) 100Mb 0ms DropTail
D
D
D
## setup the edge nodes and links
for {set i 5} {$i <= 10} {incr i} {
set node($i) [$ns node]
tb-set-node-os $node($i) FC4-STD
set j [expr $i % 5]
set link($i) [$ns duplex-link $node($i) $node($j) 1000Kb 0ms
DropTail]
tb-set-link-simplex-params $link($i) $node($i) 0ms 500Kb 0.0
}
D
D
D
# Go!
$ns rtproto Static
$ns run
winter 2008
Evaluation Tools
39
What is Emulab Good For?
(creators words)
• Simulation
– Fast prototyping, easy to use, but less realistic
• Small static testbeds
– Real hardware and software, but hard to configure and maintain,
and lacks scale
• Live networks
– Realistic, but hard to control, measure, or reproduce results
Emulab complements and also helps validate these
environments
winter 2008
Evaluation Tools
40
What is Emulab Good For?
• Studying routing protocols
– Difficult to create richly connected nodes
– Experiments that need access to the network core
• Studying peer-to-peer protocols
– But, lacks scale
• Scaling experiments with “raw” machines with no
link emulation
• Good controlled environment for testing before
going to PlanetLab
winter 2008
Evaluation Tools
41
Live Distributed Testbed
(PlanetLab)
644 nodes at 304 sites
www.planet-lab.org
winter 2008
Evaluation Tools
42
Resource Sharing
• Statistical sharing of resources
– Each application has a “slice” of the overlay resources
• Bandwidth, CPU, memory
– No guarantees on bandwidth, CPU or memory fairness
– Slices guaranteed 1Mbps of bandwidth, rest shared by all
slices
winter 2008
Evaluation Tools
43
Resource Sharing
PlanetLab
Emulab
winter 2008
Evaluation Tools
44
Slices
winter 2008
Evaluation Tools
45
Slices
winter 2008
Evaluation Tools
46
Slices
winter 2008
Evaluation Tools
47
Per-Node View
Node
Mgr
Local
Admin
VM1
VM2
…
VMn
Virtual Machine Monitor (VMM)
winter 2008
Evaluation Tools
48
PlanetLab Virtualization: VServers
• Kernel patch to mainstream OS (Linux)
• Gives appearance of separate kernel for
each virtual machine
– Root privileges restricted to activities that do not affect
other vservers
• Some modification: resource control (e.g.,
File handles, port numbers) and protection
facilities added
winter 2008
Evaluation Tools
49
How to Use PlanetLab
• Get a slice
– Request local PI
– Setup SSH keys (no passwords on PlanetLab)
• Add nodes to your slice through the web interface
• Write (or borrow) scripts to deploy and run your
system on PlanetLab and gather data from
PlanetLab
– Pssh/pscp (parallel versions of ssh/scp)
• Remember, you’re running on a live network!
– Test your code before you deploy
winter 2008
Evaluation Tools
50
Not Just a Testing
Infrastructure
• PlanetLab started by Intel …
“…to support seamless migration of an
application from an early prototype,
through multiple design iterations,
to a popular service that continues to evolve.”
winter 2008
Evaluation Tools
51
Testbed Comparison
•ns2
•Emulab
•PlanetLab
•Event-driven
simulated network
•Scale of 100s
depending of
configuration
•Simulated topology
•Controlled
environment
•Emulated network
•Scale of 10s most of
the time
•Real controllable
topology
•Controlled
environment
•Real overlay network
•Scale of 100s (goal is
over 1000)
•Real fixed topology
•Live network,
uncontrolled
environment
winter 2008
Evaluation Tools
52