Download CERNET 40G Experience

Lessons Learned
TEIN2 and CERNET
Xing Li
2007-01-22
NOC
Outline
•
•
•
•
Introduction
TEIN2 routing policy
CERNET BGP Experience
Lessons learned
NOC
Simple Case
(where BGP can handle things easily)
• Global transit
– To tier 1 or tier 2 commodity networks
• Care the aggregation
• Care the load balancing
• Don’t care the symmetry
• Peering (no transit, except for the down streams)
– To domestic ISPs (bi-literal or via IX)
• Care the business model
– To academic partners
• Care the performance
• Care the symmetry
NOC
Complicated Case
(where BGP cannot handle things easily)
• Global transit
– To tier 1 or tier 2 commodity networks
• Care the aggregation
• Care the load balancing
• Don’t care the symmetry
• Academic transit
– To multiple transit backbones within academic scope
•
•
•
•
•
Care the aggregation
Care the load balancing
Care the performance
Care the symmetry
Etc.
• Peering (no transit, except for the down streams)
– To domestic ISPs (bi-literal or via IX)
• Care the business model
– To academic partners
• Care the performance
• Care the symmetry
NOC
Two Steps to Implement the Policy
• Identification
– IP prefix
– AS path regular expression
– Community tag
• Path selection
– AS path (inbound and outbound)
– Local-preference (outbound)
– More specific (inbound)
NOC
For Transit Network
TEIN2 Example
NOC
TEIN2 Topology
NOC
The Principle of Routing Design
for the TEIN2 network
• To provide interconnection among TEIN2 partners and
between TEIN2 partners and EU NRENs.
• To provide back-up paths within the TEIN2 network
and/or via partner networks for service resilience when
possible.
• To provide a flexible and transparent routing policy to
TEIN2 NRENs.
• To avoid being selected by GÉANT, Abilene and other
R&E networks outside TEIN2 as the preferred transit
network.
• To minimize the adjustment of the external peers’ routing
policy outside TEIN2 networks, e.g. GÉANT and APAN.
NOC
TEIN2 Routing Policy
• Enable additive community tagging to mark
the prefix announcements.
• Adopt AS number prepending as the preferred
BGP policy for TEIN2 traffic adjustment within
TEIN2 backbone.
– Use ingress AS number prepending for outbound
traffic adjustment, including traffic from TEIN2 POP to
NRENs, GÉANT and APAN.
– Use egress AS number prepending for inbound traffic
adjustment, including traffic from NRENs, GÉANT and
APAN to TEIN2 POP.
• May use Local-Preference amendment as the
last resort of mechanism for fine tuning on
TEIN2 traffic over the backbone.
NOC
For NRN
CERNET Example
NOC
CERNET Topology
NOC
CERNET Peering
3G
Internet
CERNET 2
CERNET
Domestic
Peering
12G
10G
CNGI
Peering
DRAGONTAP
CNGI-BJIX
DRAGONLIGHT
155M
100M
1G
155M
2x155M
622M
HARNET
TEIN2
ASNET
APAN
KOREN
STARLIGHT
NOC
CERNET Routing Policy
• Outbound
– Use AS number prepending if possible
– Heavily use Local-Preference
– Enable additive community tagging to mark
the prefixes
• Inbound
– Use AS number prepending if possible
– Announce more specifics
– Enable additive community tagging to mark
the prefixes
NOC
Case 1
TAIWAN Earthquake
NOC
Earthquake on 26th DEC 2006
NOC
Why did not include this policy
before the earthquake?
NOC
Case 2
Routing and End-to-end
performance
NOC
Ping and dvping beacons
NOC
Here in the APAN venue WLAN
NOC
Lessons Learned (1)
• The nature of BGP is reachability
– Stupid routing happen
– Policy based routing makes thing very complicated
– The routing and topology are very dynamic environment
• The key words are: simple, open and controllability
– For transit network
• Simple
• Open
– For NRN
• Simple
• Controllability
• Why did not include this policy before the earthquake?
– Because it is a NP problem and there are many contradict
requirements
– Mission impossible
– What should be the solution?
NOC
Lessons Learned (2)
• It seems that we still need to do a lot
manual BGP policy adjustment, case by
case with the help of
– Multi-site collaborations
– Routeviews
• We have to compare the routing table with
the end-to-end performance matrix
– dvping tool
NOC