Download Slides - TERENA Networking Conference 2002

Abilene
and Internet2 Engineering Update
Guy Almes <[email protected]>
Terena Networking Conference 2002
Limerick, Ireland
Outline
 Abilene Update
 Engineering Update
»
»
»
»
»
Multicast
IPv6
QoS
End-to-End Measurements
Transport for Bulk Data Flows
6 May 2002
2
Internet2 Engineering Objectives
 Provide our universities with superlative
networking:
» Performance
» Functionality
» Understanding
 Make superlative networking strategic for
university research and education
6 May 2002
3
Abilene Update
Current 2.5 Gb/s Abilene Network
Plans for 10 Gb/s Upgrade
6 May 2002
4
Abilene is a Partnership
To build/operate Abilene, Internet2
partners with:
»Cisco Systems (routers, switches, and access)
»Juniper Networks (routers)
»Nortel Networks (SONET kit)
»Qwest Communications (circuits and collocation)
»Indiana University (network operations center)
»Internet2 Test & Evaluation Centers (ITECs)
– North Carolina
– Ohio
6 May 2002
5
Current Abilene Status
IP-over-SONET backbone (2.5 Gb/s)
» 53 direct connections
» 4 2.5 Gb/s connections
» 1 Gigabit Ethernet trial
» 23 will connect via at least 622 Mb/s by 1Q02
» Number of ATM-based connections decreasing
215 participants: universities and labs
» All 50 states, District of Columbia, and Puerto Rico
» 15 regional gigaPoPs support ~70% of participants
Expanded access
» 50 sponsored participants
» 23 state education networks (SEGPs)
6 May 2002
6
Abilene international connectivity
Transoceanic R&E bandwidths growing !?
» GÉANT: 5 Gb/s between Europe and New York City
Key international exchange points:
» StarTap and StarLight: Chicago (GigE)
» AmPath: Miami (155 Mb/s ATM)
» Pacific Wave: Seattle (GigE)
» MAN LAN: New York City (GigE/10GigE planned)
» CA*net3/4: Seattle, Chicago, and New York
» CUDI: CENIC and Univ Texas El Paso
International transit service
» Collaboration with CA*net3 and StarTap
6 May 2002
7
09 March 2002
Abilene International Peering
STAR TAP/Star Light
Pacific Wave
AARNET,
APAN/TransPAC,
CA*net3,
TANET2
APAN/TransPAC, Ca*net3, CERN, CERnet, FASTnet, GEMnet,
IUCC, KOREN/KREONET2, NORDUnet, RNP2, SURFnet,
SingAREN, TAnet2
NYCM
SNVA
BELNET,
CA*net3,
Washington
GEANT*,
HEANET,
JANET,
NORDUnet
GEMNET,Sacramento
SINET,
SingAREN, WIDE
LOSA
Los Angeles
UNINET
OC3-OC12
San Diego (CALREN2)
CUDI
El Paso (UACJ-UT El Paso)
CUDI
AMPATH
REUNA, RNP2 RETINA,
ANSP, (CRNet)
* ARNES, CARNET, CESnet, DFN, GRNET, RENATER, RESTENA, SWITCH, HUNGARNET, GARR-B, POL-34, RCST, RedIRIS
Packetized Raw HDTV
Raw HDTV/IP: single 1.5 Gb/s UDP flow
DARPA-funded project of USC/ISIe, Tektronix, and
Univ Washington
» 6 Jan 2002: Seattle to Washington DC via Abilene
» 18 hours: no packets lost, 15 resequencing episodes
» End-to-end network performance (includes P/NW & MAX)
– Loss: <0.8 ppb (90% c.l.)
– Reordering: 5 ppb
» Transcontinental 1-Gb/s TCP
requires loss of
– <30 ppb (1.5 KB frames)
– <1 ppm (9 KB jumbo)
6 May 2002
9
End-to-End Performance:
‘High bandwidth is not enough’
Bulk TCP flows
»Current median flow over Abilene: 1.9 Mb/s
– 95th percentile: 7.0 Mb/s
6 May 2002
10
Future of Abilene
Internet2/Qwest agreement amended,
and extended to Oct-06
Upgrade now underway to shift
»from OC-48c (using Nortel OC-192 Sonet)
»to 10-Gb/s lambda (unprotected)
»x4 increase in core backbone bandwidth
6 May 2002
11
Next generation Abilene router
selection
Extensive router specification and testing
Tests focused on next gen advanced services
» High performance TCP/IP throughput
» High performance multicast
» IPv6 functionality and throughput
» Classification for QoS and measurement
3 router platforms tested and commercial ISPs
referenced
Juniper T640 platform selected
6 May 2002
13
Deployment timing
Ongoing: Backbone router procurement,
detailed deployment planning
July: Rack assembly (Indiana Univ.)
Aug/Sep: New rack deployment
Fall: First Wave lambdas commissioned
Fall meeting demonstration events
» Internet2 Fall Member Meeting (Los Angeles): late Oct.
» SC2002 (Baltimore): mid Nov.
2003: Remaining lambdas commissioned
6 May 2002
14
Key next-gen Abilene emphases: I
Native IPv6
» Motivations
– Resolving IPv4 address exhaustion issues
– Preserving original End-to-End Architecture model
– International collaboration
– Router and host OS capabilities
» Run native IPv6, concurrent with IPv4
» Replicate Abilene’s 1999 multicast deployment strategy
» Close collaboration with Internet2 IPv6 Working Group
6 May 2002
15
Key next-gen Abilene emphases: II
Network resiliency
»Abilene lambdas will not be ring protected
»Increasing use of videoconferencing/VoIP impose
tighter restoration requirements (<100 ms)
»Options:
– MPLS/TE fast reroute (initially)
– IP-based IGP fast convergence (preferable)
6 May 2002
16
Key next-gen Abilene emphases:
III
Deeper measurement capabilities
» Significant factor in NGA rack design
– 4 dedicated servers at each nodes
– Additional provisions for future servers
– Local data collection to capture data at times of network
instability
» Enhance active probing
– Now: Latency and jitter, loss, reachability (Surveyor)
– Regular TCP/UDP throughput tests: ~1 Gbps
• Separate server for E2E performance beacon
» Enhance passive measurement
– Now: SNMP (NOC) and traffic matrix/type (Netflow)
– Routing (BGP and IGP)
– Optical splitter taps on backbone links at select location(s)
6 May 2002
17
Abilene Observatories
Currently a sketch of a program for better support of
computer science research
1) Improved and accessible data archive
» Need coherent database design
» Unify & correlate 4 separate data types
– SNMP, active measurement data, routing, Netflow
2) Provision for direct network measurement and
experimentation
» Resources reserved for two additional servers
– Power (DC), rack space (2RU), router uplink ports (GigE)
» archive/measurement/experiment
6 May 2002
18
Abilene Upgrade Summary
Backbone upgrade project underway
» Partnership with Qwest extended through 2006
» Juniper T640 routers selected for backbone
» 10-Gb/s backbone lambda deployment starts this fall
Advanced service foci
» Native, high-performance IPv6
» Enhanced, differentiated measurement
» Network resiliency
Incremental, non-disruptive transition
6 May 2002
19
Multicast
 By 1998,
»
»
»
»
Routing protocols existed
Deployment of native IP multicast quite rare
Early MBone no longer scalable
Considered key to new conferencing and streaming applications
 Current native multicast support
» PIM-Sparse, MBGP, and MSDP
 Emphases on
» Deployment and support for operations
» Applications
» Working to make it scalable
6 May 2002
20
6 May 2002
21
Current Multicast Emphases
Pressing ahead on Deployment
»What are the current inhibitors to progress?
Applications / Content
»Make it useful for your campuses
»Explore the role of multicast in the future Internet
Improve Scalability
»Press deployment of SSM
»Explore the role of SSM
6 May 2002
22
Could SSM be Enough?
 'Classic' Multicast
»
»
»
»
Group <g> has global significance
A user creates, joins, sends to g
Others can join, then send to and/or listen to g
MBGP, PIM-SM, MSDP triad
 Source Specific Multicast
»
»
»
»
Group <g> has local significance
A user 's' creates, sends to <s,g>
Others can subscribe to, then listen to <s,g>
No need for MSDP (or allocation of <g> values)
6 May 2002
23
Implications of SSM
 Simplify Multicast Routing / Addressing
» No need for global class-D address allocation
» No need for source discovery
 Complicates 'few-to-few' applications
» Define all the members of the application-level group
» Both a burden and an opportunity
 Allows better Security, Scalability
 Requires new version of IGMP
6 May 2002
24
IPv6
 Clarify motivation for IPv6
» End-to-end transparency and global addressability
» Supports application innovation, e.g., peer-to-peer
 Support deployment and engineering expertise on
networks, especially on campus
 Anticipate need for first-class support
» E.g., 10 Gb/s Abilene upgrade
» E.g., Linux, Windows XP
6 May 2002
25
6 May 2002
26
Current IPv6 Emphases
IPv6 Training Workshops
» About 8-10 workshops this year
» First: in Los Angeles, hosted by CENIC, in February
Get some IPv6 on each campus/gigaPoP
Prepare for native peering
» Abilene to gigaPoP
» gigaPoP to campus
» continue within campuses to key departmental LANs
Explore applications, DNS, operational stability
6 May 2002
27
QoS
Premium Service Retrospective
»Inter-AS Premium Service proved too ambitious
»Too great a demand on all routers to be able to police and
to shape
»Too great a demand on ability of peering networks to
coordinate
Implicit claims
»Over-provisioning plus removal of non-congestive loss
»Adaptive applications
6 May 2002
28
Non-Elevated Services
Scavenger
»Less than best effort
»Easy to deploy
»Applications:
– Massive file transfers
– Marking non-performance-sensitive applications
Alternative Best Efforts
»Active area of research
»Avoid gaming by users, while avoiding need for policing
6 May 2002
29
Localized Elevated Services
Nature of Congestion as a threat
»Less on national/international backbones
»More at hard-to-upgrade local/metro networks
Decentralized Experimentation
Coordinates with some Scavenger
deployment
Active area of study by the working group
6 May 2002
30
The Current Situation
 Our universities have access to an
infrastructure of considerable capacity
» examples of multi-hour 1.6 Gb/s flows with no loss and very
little reordering
 End-to-end performance varies widely
» but 40 Mb/s flows not always predictable
» users don't know what their expectations should be
 A well-known mismatch
6 May 2002
31
What are our Aspirations?
 Candidate Answer #1:
Switched 100BaseT + Well-provisioned
Internet2 networking at 80 Mb/s
 But user expectations and experiences vary
widely
6 May 2002
32
What are our Aspirations?
 Candidate Answer #2:
Lower user expectations and minimize
complaining phone calls
 There is a certain appeal I suppose...
6 May 2002
33
What are our Aspirations?
 Candidate Answer #3:
Raise expectations, encourage aggressive
use, deliver on performance/functionality to
key constituencies.
 Not the easy way, but necessary for success
6 May 2002
34
Threats to
End to End Performance
 Fiber problems
» dirty fiber
» dim lighting
» 'not quite right' connectors
6 May 2002
35
Threats to
End to End Performance
 Fiber problems
 Switches
» horsepower
» full vs half-duplex
» head-of-line blocking
6 May 2002
36
Threats to
End to End Performance
 Fiber problems
 Switches
 Inadvertently stingy provisioning
» mostly communication
» happens also in international settings
6 May 2002
37
Threats to
End to End Performance
 Fiber problems
 Switches
 Inadvertently stingy provisioning
 Wrong Routing
» asymmetric
» best use of Internet2
» distance
6 May 2002
38
Threats to
End to End Performance




Fiber problems
Switches
Inadvertently stingy provisioning
Wrong Routing
 Host issues
» NIC
» OS / TCP stack
» CPU
6 May 2002
39
Perverse Result
 'Users' think the network is congested or that
the Internet2 infrastructure cannot help them
 'Planners' think the network is underutilized,
no further investment needed, or that users
don't need high performance networks
6 May 2002
40
Transport Protocol Issues
Improved TCP Implementation
»Web100 Project
»SACK, Window Scaling
»ECN
But, still subject to fundamental limits
»Mathis et al. Theoretical result
»TCP-throughput = (C x MTU) / (RTT x sqrt(loss))
»Prospects for raising MTU, reducing RTT, loss
6 May 2002
41
Need to supplement TCP
Bulk Data Transfers
»Absolute need to include congestion control
»But include flow control in a more aggressive way
Related Ideas
»Transport-level gateways?
»Known-Capacity pipes vs Groping for available Capacity
6 May 2002
42
www.internet2.edu