Download ppt

Internet Routing (COS 598A)
Today: Interdomain Traffic Engineering
Jennifer Rexford
http://www.cs.princeton.edu/~jrex/teaching/spring2005
Tuesdays/Thursdays 11:00am-12:20pm
Outline
• Border Gateway Protocol
– BGP protocol
– BGP policies
– BGP decision process
• BGP traffic engineering for outbound traffic
– Predicting effects of policy changes
– Identifying good policy changes
• What about inbound traffic?
– AS prepending and MEDs
– Inter-AS negotiation
Interdomain Routing: Border Gateway Protocol
• ASes exchange info about who they can reach
– IP prefix: block of destination IP addresses
– AS path: sequence of ASes along the path
• Policies configured by the AS’s operator
– Path selection: which of the paths to use?
– Path export: which neighbors to tell?
“12.34.158.0/24: path (2,1)”
3
“12.34.158.0/24: path (1)”
1
2
data traffic
data traffic
12.34.158.5
Components of BGP
• BGP protocol
– Definition of how two BGP neighbors communicate
– Message formats, state machine, route attributes, etc.
– Standardized by the IETF
• Policy specification
– Flexible language for filtering and manipulating routes
– Indirectly affects the selection of the best route
– Varies across vendors, though constructs are similar
• BGP decision process
– Complex sequence of rules for selecting the best route
– De facto standard applied by router vendors
– Being codified in a new RFC for BGP coming soon
BGP Protocol: BGP Sessions
Establish session on
TCP port 179
AS1
BGP session
Exchange all
active routes
AS2
Exchange incremental
updates
While connection
is ALIVE exchange
route UPDATE messages
BGP Protocol: Update Messages
• Update messages
– Advertisement
• New route for the prefix (e.g., 12.34.158.0/24)
• Attributes such as the AS path (e.g., “2 1”)
– Withdrawal
• Announcing that the route is no longer available
• Numerous BGP attributes
– AS path
– Next-hop IP address
– Local preference
– Multiple-Exit Discriminator
–…
BGP Policy: Applying Policy to Routes
• Import policy
– Filter unwanted routes from neighbor
• E.g. prefix that your customer doesn’t own
– Manipulate attributes to influence path selection
• E.g., assign local preference to favored routes
• Export policy
– Filter routes you don’t want to tell your neighbor
• E.g., don’t tell a peer a route learned from other peer
– Manipulate attributes to control what they see
• E.g., make a path look artificially longer than it is
BGP Policy: Influencing Decisions
Open ended programming.
Constrained only by vendor configuration language
Receive Apply Policy =
filter routes &
BGP
Updates tweak attributes
Apply Import
Policies
Based on
Attribute
Values
Best
Routes
Best Route
Selection
Best Route
Table
Apply Policy =
filter routes &
tweak attributes
Apply Export
Policies
Install forwarding
Entries for best
Routes.
IP Forwarding Table
Transmit
BGP
Updates
BGP Decision Process: Path Selection on a Router
• Routing Information Base
– Store all BGP routes for each destination prefix
– Withdrawal message: remove the route entry
– Advertisement message: update the route entry
• Selecting the best route
– Consider all BGP routes for the prefix
– Apply rules for comparing the routes
– Select the one best route
• Use this route in the forwarding table
• Send this route to neighbors
BGP Decision Process: Multiple Steps
• Highest local preference
– Set by import policies upon receiving advertisement
• Shortest AS path
– Included in the route advertisement
• Lowest origin type
– Included in advertisement or reset by import policy
• Smallest multiple exit discriminator
– Included in the advertisement or reset by import policy
• Smallest internal path cost to the next hop
– Based on intradomain routing protocol (e.g., OSPF)
• Smallest next-hop router id
– Final tie-break
BGP Decision Process in Action
“(2, 1)”
“(3, 4, 1)”
“(2, 1)”
But, what if the path “(3,4,1)” would be better?
Manipulating Policy to Move the Traffic
• Assign local preference to…
– Prefer one neighbor over another for a prefix
– Prefer certain AS paths over others
• Router configuration languages
– Specifying rules for setting local-pref attribute
– “if path(3, *, 1), then local-pref=110”
– “else, local-pref=100”
• Allow policy to over-ride shortest AS path
– Indirect way of making one path look better or
worse than another
– Main way to do BGP traffic engineering today
BGP Traffic Engineering
BGP Traffic Engineering
• Limitations of intradomain traffic engineering
– Alleviating congestion on edge links
– Making use of new or upgraded edge links
– Influencing choice of end-to-end path
• Extra flexibility by allowing changes to BGP policies
– Direct traffic toward/from certain edge links
– Change the set of egress links for a destination
2
4
1
3
BGP Modeling for Traffic Engineering
• Predict effects of changes to import policies
– Inputs: routing, traffic, and configuration data
– Outputs: flow of traffic through the network
Topology
eBGP
routes
BGP policy
configuration
BGP routing
model
Offered
traffic
Flow of traffic through the network
Steps in the Model
• Effects of import policy on the routes
– Inputs: BGP routes, and import policies
– Output: BGP routes modified by policies
• Set of best routes for the network
– Inputs: BGP routes modified by policies
– Output: set of best routes (max local-pref, min AS path,…)
• Best route per router
– Inputs: set of BGP routes, and intradomain costs
– Outputs: best route (closest egress) per prefix per router
• Link-level paths
– Inputs: best route per router, and intradomain costs
– Outputs: link-level shortest path(s) from entry to egress
But, Hard to Select Good Import Policy Changes
• Can’t enumerate all choices
– Many eBGP sessions
– Around 100K prefixes
– Highly programmable policies
• Risk of creating side-effects
– Overhead of BGP routing changes
– Unpredictable behavior of others
• Vulnerability to changes
– New eBGP routes from neighbors
– Shifts in the traffic volume per prefix
Traffic Engineering Guidelines
• Predictability
– Ensure the BGP decision process is deterministic
– Assume that BGP updates are (relatively) stable
• Outbound traffic (import policy, local preference)
– Easier to control how traffic leaves the network
– Cooperate with neighbor ASes for inbound traffic
• Limit overhead introduced by routing changes
– Minimize frequency of changes to routing policies
– Limit number of prefixes affected by changes
• Limit impact on how traffic enters the network
– Avoid new routes that might change neighbor’s mind
– Select route with same attributes, or at least path length
Measurement Data
• Analysis of peering links
– Links connecting AT&T to other large providers
– Relatively small # of high-end routers
• BGP routing tables
– Log daily output of “show ip bgp” command
– Extract BGP routes for each prefix
• Cisco Netflow
– Collect continuous flow-level measurement
– Aggregate the traffic by prefix
– Focus on outbound traffic to peers
Controlling the Scale
• Destination prefixes
– More than 90,000 destination prefixes
• Don’t want to have per-prefix routing policies
– Small fraction of prefixes contribute most of the traffic
• Focus on the small number of heavy hitters
– Define routing policies for selected prefixes
• Routing choices
– About 27,000 unique “routing choices”
• Help in reducing the scale of the problem
– Small fraction of “routing choices” contribute most traffic
• Focus on the very small number of “routing choices”
– Define routing policies on common attributes
Avoid Impacting Downstream Neighbors
Will traffic volume change???
Predictable Routing Change
• Predictability
– Do not change the route sent to downstream neighbor
– Focus on prefixes where all “best” routes are identical
– Neighbors do not even receive a new BGP advertisement!
• Example application
– Multiple links to same peer, with one congested link
– Assign lower local-pref at that link for some prefixes
• Empirical results
– 83.5% of prefixes have shortest AS paths that are all
identical (same next-hop AS, same AS path, etc.)
– These prefixes are responsible for 45% of the traffic
– Plenty of scope to move traffic in a predictable fashion
Semi-Predictable Routing Change
• Semi-predictability
– Do not change the length of the AS path sent to neighbor
– Neighbors receive new advertisement with same length
– (Hopefully) they still make the same routing decision
• Example application
– Need to move some traffic from one peer to another
– Find prefixes with “best” paths via both neighbors
– Assign lower local-pref at some links for some prefixes
• Empirical results
– 10-15% of prefixes have shortest paths with 2 next hops
– These prefixes contribute 35-40% of the traffic
– Potential to move traffic in a semi-predictable fashion
Influence of AS Path Length
• AS path length
– Plays a significant role in the BGP decision process
– All “best” routes must have the same AS path length
– 10% of prefixes have choices with different path lengths
• An idea: a more flexible approach
– Possible to disable consideration of AS path length, and
incorporate AS path length in local-pref assignment
– E.g., treat paths of length 3 and 4 as equally good
• AS prepending by other ASes
– Inflating AS path length by adding fake hops
– E.g., “701 80 80 80” instead of “701 80”
– 18% of routes had some form of AS prepending
Inbound Traffic
Why Inbound Traffic is Hard to Manage
• Other ASes decide how to send to you
– Destination-based routing
– Other ASes decide which path to take
– Based on their own policies
2
4
1
p
3
AS 2 doesn’t know how AS 1 will send traffic toward p
AS Prepending
• Artificial inflate AS path length
– Prepend your own AS in the path
– E.g., turn “3 4 5” into “3 3 3 4 5”
– Hope to make the path less attractive
1
“3 4 5”
3
“3 3 3 4 5”
Multiple Exit Discriminator (MED)
• Tell your neighbor what you want
– MED attribute to indicate receiver preference
– Decision process picks route with smallest MED
– Can use MED for “cold potato” routing
– But, have to get your neighbor to accept MEDs
1
“3 4 5” with MED=1
3
“3 4 5” with MED=2
Inter-AS Negotiation
• Better to cooperate?
– Negotiate where to send
– Inbound and outbound
– Mutual benefits
Customer B
Provider B
multiple
peering
points
• But, how to do it?
Early-exit
routing
– What info to exchange?
– How to prioritize the
many choices?
– How prevent cheating?
• Open research territory
Provider A
Customer A
Next Time: Multi-Homed Customers
• Two papers
– “A Measurement-Based Analysis of Multi-homing”
(SIGCOMM’03)
– “Optimizing Cost and Performance for
Multihoming” (SIGCOMM’04)
• Reviews of both papers
– Brief summary of the paper
– Reasons to accept the paper
– Reasons to reject the paper
– Suggestions for future research directions