Download 20060718-bpc

Backbone Performance
Comparison
Jeff Boote, Internet2
Warren Matthews, Georgia Tech
John Moore, MCNC
1
Overview
• We (in NC) were asked to compare the relative
performance of various IP service providers
 Interest from both local CIOs and Internet2
• We decided to measure relative end-to-end
latency and jitter
• Recruited a few other ITECs (Ohio and Texas)
and GA Tech to help
• Jeff Boote got interested since we were using
owamp
2
Method
• Setup owamp machine at each site with multiple
interfaces per NIC
• Use host routes to force traffic to a specific
destination via a specific provider
• Create a mesh of these running continuously
and dump results to a database
• Add traceroute information to verify paths and
look for routing changes
3
Path types
• Path will vary
depending on
whether source
and destination
sites share
provider, or not.
• Doesn’t take
“natural” or policy
routing into
consideration,
but useful for
comparative
purposes.
4
As we progressed…
• New paths became available…
 VPLS (Layer 2 VLAN) between three of the ITECs (NC, OH and
TX)
 Described in sidebar
 NLR PacketNet between NC and GT
 Not all that interesting, since both sites attach to the same NLR
router in Atlanta
• Added NLR to new interface on same NIC, added VPLS
to a separate NIC on the same machines
• TAMU site setup and running, but o good data available
yet
 Had to remove host routes due to other routing changes going
on locally
5
Available Data from OWAMP
•
•
•
•
•
•
Latency
Latency variation (jitter ~ 95%-min)
TTL (num hops)
Duplicates
Loss
Reordering (not likely at 1 pps)
6
OWAMP “sender”
configuration
Only last router before “backbone” shown
• Each host has multiple
virtual addresses
configured (one per
“network”)
• Continuous stream of
packets (1 pps - exp dist.)
per network address
“pair”
• Traffic is directed onto
specific network based
on dest address
7
LATAB
(Traceroute when source is routed through Abilene)
OH
nms4-ipls
IPLS
CHIN
NYCM
KSCY
NC
WASH
nms4-hstn
HSTN
ATLA
TAMU
GT
nms4-wash
8
LATQW
(Traceroute when source is routed through Qwest)
CHI-EDGE
CHI-CORE
DCA-CORE
DCA-EDGE
NC
ATLA-CORE
ATLA-EDGE
OH
GT
9
LATL3
(Traceroute when source is routed through Level3)
Qwest
Washington
Washington
Washington
Asymmetric routing:
Northbound via Charlotte
Southbound via Raleigh.
Raleigh
Unknown
NC
Charlotte
Charlotte
Atlanta
OH
Unknown
Unknown
GT
ATLAL3
10
LATO3
(Traceroute when source is routed through another provider - GT/Cogent)
OH
Qwest
NC
CORE
ATLA
GT
11
LATNLR
ATLA
NC
GT
12
LATVPLS
OH
NC
TAMU
13
Preliminary Results
• Small amount of data collected so far
• Working on how best to visualize
combination of pieces (latency, loss,
routing changes, etc.)
 Looking for “stability” metric (but stability is
application dependent)
• More analysis needed
14
Loss overview
15
NC to GT
•
NLR is lower latency.
 This is expected as GT and
NC are connected to the
same router.
 NC connection is backhauled
via NLR L2 service.
Qwest
NLR
•
Qwest and Abilene go via
Washington.
 The long way…
•
Level3
Abilene
For the Level3 path, there is
an unidentified hop just before
the GT campus.
 Rate limiter?
 Expected NLR and Level3
paths to be closer
16
GT to NC
NLR
Qwest
Cogent
Level3
Abilene
• NLR and Level3
paths similar
• Cogent hands off to
Qwest to get to NC
17
Latency Range
NC to GT
14
Range of Latency Between NC and GT
12
10
Latency (ms)
Level3 via
Raleigh
8
6
4
Level3 via
Charlotte
2
Input to GT is always longer?
18
0
AB_GT_NC
AB_NC_GT
L3_GT_NC
L3_NC_GT
QW_GT_NC
QW_NC_GT
NLR_GT_NC
NLR_NC_GT
NC to OH
Qwest
Level3
• Marginally quicker
across Qwest (via
Washington and
Chicago).
• Abilene via New York,
Chicago and
Indianapolis.
Abilene
19
OH to NC
•
Qwest
Abilene
OH doesn’t use Level3,
so no return path to NC
via Level3
20
Latency Range
NC to OH
22
Latency Range Between NC and OH
21
Latency (ms)
20
No return path for L3_NC_OH
19
18
17
21
16
AB_OH_NC
AB_NC_OH
L3_OH_NC
QW_OH_NC
QW_NC_OH
GT to OH
Abilene
Cogent
• Abilene more direct
via Indianapolis
• Qwest via Chicago
• Cogent, Level3 hand
off to Qwest
Qwest
Level3
22
OH to GT
•
Qwest
OH doesn’t use Level3,
so no return path to GT
via Level3
Abilene
23
Latency Range
GT to OH
30
Latency Range Between GT and OH
25
No return path for L3_GT_OH.
Latency (ms)
20
15
10
5
24
0
AB_GT_OH
AB_OH_GT
L3_OH_GT
QW_GT_OH
QW_OH_GT
Summary
• From a latency perspective, topology is the
overriding parameter
 So far we’re not seeing huge latency deltas between
R&E and commodity between two endpoints
• Loss in commodity networks is pretty good
 They’ve improved in the last 10 years
• Looking for a quality metric (stability?) to
combine the things we can measure
25
VPLS Sidebar
• Virtual Private LAN Service - multipoint
Ethernet service over IP/MPLS backbone
• Created between ITECs as overlay on
Abilene
• PE routers sit in GigaPoP address space,
interconnected via interdomain LSPs
• Abilene T640s are P routers
26
VPLS Overview
•
•
•
Full Mesh of LSPs
BGP for inter-PE communication
Ethernet encapsulation at PE-CE
27
View from Ohio
To NC
To TX
No routers!
28
View from NC PE
Local NC MAC
address
TX MAC
address
OH MAC
address
29