Download Peering Planning Cooperation without Revealing Confidential

Peering Planning Cooperation without
Revealing Confidential Information
Thomas Telkamp
telkamp at cariden.com
RIPE 52
Istanbul, Turkey
www.cariden.com
RIPE 52
(c) cariden technologies
1
The Issue
• Multi-Homed Neighbor, 2 or more links > 50%
• Example
– 1000Mbps connections to Peer X in 3 locations
– SJC-to-Peer = 600Mbps, NYC = 100, WDC = 600
– SJC-to-Peer link fails
• Are we in trouble?
...
or
...
?
SEA
SJC
X
KCY
1
HST
LAX
Peer X
RIPE 52
BOS
CHI
00
M
s
p
b
?
SEA
BOS
CHI
NYC
WDC
0
0
g
ATL
2
n
1 o
(c
t
s
e
MIA
NYC
)
d
e
SJC
X
KCY
7
HST
LAX
Peer X
00
WDC
ATL
00
6
MIA
2
Capacity Planning Utopia
• Uniform capacity links
• Diverse connections
(unlikely double failures at Layer 3)
• Upgrade at 50%
(planning objective is to be resilient to single
failures)
RIPE 52
3
Capacity Planning Reality
• Range of capacities
• Multiple Layer 3 failures
• Upgrade impediments (money, cable plant, ...)
RIPE 52
4
IGP Different from BGP
• Data is more accessible
• Failure behavior is predictable
• Established process for within AS planning
– Gather Data
• Topology (OSPF, IS-IS, ...)
• Traffic matrix
[1]
• Estimate growth
– Simulate for failures
– Perform traffic engineering (optional)[2]
– Upgrade as necessary
• Commercial and free tools
RIPE 52
5
The Trouble with BGP
• Data is larger and harder to access
• BGP decision process complicated
• Planning practices not well established
• Failure behavior often depends on someone
else’s network!
subject of
– e.g., incoming traffic from a peer
RIPE 52
this talk
6
BGP Path Decision Algorithm[1]
1. Reachable next hop
2. Highest Weight
3. Highest Local Preference
4. Locally originated routes
5. Shortest AS-path length
6. IGP > EGP > Incomplete
Respect MEDs
7. Lowest MED
8. EBGP > IBGP
Shortest Exit Routing
9. Lowest IGP cost to next hop
10. Shortest route reflection cluster list
11. Lowest BGP router ID
12. Lowest peer remote address
[1] Junos algorithm shown here. Cisco IOS uses a slightly different algorithm.
RIPE 52
7
Common Routing Policies
• Shortest Exit
– Often used for sending to peers
– Get packet out of network as soon as possible
– Local Prefs used to determine which neighbor,
IGP costs used to determine which exit
• Respect MEDs
– Often used for customers who buy transit
– Deliver packets closest to destination
– Neighbor forwards IGP costs as MEDs
(multi-exit discriminators)
RIPE 52
8
Blind Spots
• Cannot predict behavior when routing depends
on other network (see 3 cases below).
Relationship to
Remote AS
Routing To
Remote AS
Shortest Exit in
Peer known network
Respect MEDs
Customer from unknown
Transit Shortest Exit in
Provider known network
RIPE 52
Routing From
Remote AS
Shortest Exit in
unknown network
Shortest Exit in
unknown network
Respect our MEDs
9
Failover Matrices
• Solution to peering planning blind spots
• Procedure
– Gather data
• Topology, Traffic, Routing Configurations
– Simulate knowable effects
• Generate Failover Matrices
– Share Failover Matrices for unknowables
• e.g., peer gives failover matrix for traffic it delivers, we
provide peer failover matrix for traffic we deliver
• Both sides benefit from cooperating
• AS-Internal information is kept confidential
RIPE 52
10
Failover Matrix Example
Traffic:
%Traffic:
no failure fail_SJC
ar1.sjc:Gig3/2
600
Node:Interface
ar1.nyc:ge-2/1
100
48%
(388)
ar2.wdc:ge-2/2
600
52%
(912)
%Traffic: %Traffic:
fail_nyc
fail_wdc
10% (610)
1% (606)
70%
(670)
95%
(670)
-
Note: 388Mbps=100Mbps+(0.48*600Mbps), 912=600+(0.52*600), ...
RIPE 52
11
Failover Example (from real network)
e
l
a
c
S
o
N
Peer Circuit 1: Traffic levels at five minute intervals
e
l
a
c
S
o
N
Peer Circuit 2: Traffic levels at five minute intervals
e
l
a
c
S
o
N
Peer Circuit 3: Traffic levels at five minute intervals
e
l
a
c
S
o
N
Peer Circuit 4: Traffic levels at five minute intervals
• Circuit 2 fails.
Traffic shifts to circuit 4.
RIPE 52
12
Failover Example (from real network)
e
l
a
c
S
o
N
Peer Circuit 1: Traffic levels at five minute intervals
e
l
a
c
S
o
N
Peer Circuit 2: Traffic levels at five minute intervals
e
l
a
c
S
o
N
Peer Circuit 3: Traffic levels at five minute intervals
e
l
a
c
S
o
N
Peer Circuit 4: Traffic levels at five minute intervals
• Circuit 1 fails. Some traffic shifts to 2 & 4
• Some “leaks” to other AS’s
RIPE 52
13
Questions
• How do I calculate a failover matrix?
• How do I use a failover matrix from a peer?
• What if my peer does not cooperate?
• What if a substantial amount of traffic “leaks”
to another AS?
RIPE 52
14
Calculating Failover Matrices
• Accurate and Detailed[3,4]
– Per prefix routing and traffic statistics
– Full BGP simulation
• Simple and Good Enough
– Traffic matrix based on ingress-egress pairs
• e.g., Peer1.LAX-AR1.CHI (measure and/or estimate)
instead of 192.12.3.0/24-208.43.0.0/16
– Limited simulation model
• Shortest Path, Respect MEDs
• “Our” AS plus immediate neighbors
RIPE 52
15
Using Failover Matrix from Peers
• Peer calculates failover matrix
• Peer exports failover matrix
using IP addresses of peering
links
• We import failover matrix
• We include in a representative
model of peer network
• Use Failover Matrix in
simulation
RIPE 52
16
Estimate if Peer not Cooperate
bbs1
sel2
sea1
bhx1
lba1
min1
sfo1
nrt4
roc1
← Group own
sources based on
exit location
(4 groups here)
man1
det1
cph1
nqt1
bos1
ewr1
nrt1
chi1
osa1
sjc2
pao2
tpe3
jfk1
ham1
lon3
sac1
kcy1
cle1
nyc2
wsh1
ams2
fra2
snv2
kul1
lax1
cdg2
den2
dal1
phi1
wdc2
ams1
sin1
lin1
bru1
hkg1
atl1
phx1
sna1
hou1
tpa1
mia1
bbs1
sel2
pty1
mex1
bhx1
gru1
syd1
sea1
lba1
eze1
det1
min1
sfo1
nrt4
→ Quantify shift (to 3
groups) after failure
Assume similar for
other side
man1
cph1
nqt1
roc1
bos1
ewr1
nrt1
chi1
osa1
sjc2
lon3
sac1
pao2
tpe3
ham1
jfk1
kcy1
cle1
nyc2
wsh1
ams2
fra2
snv2
kul1
lax1
cdg2
den2
hkg1
dal1
phi1
wdc2
ams1
sin1
bru1
lin1
atl1
phx1
sna1
hou1
tpa1
mia1
pty1
mex1
gru1
syd1
eze1
• Valid if topology and traffic distributions are similar
RIPE 52
17
Leaks to Other AS’s
x
A
INTERNET
B
• Simple option
– Leaks between peers relatively small
• Ignore
– Shifts between transit providers can be large
• Equal AS-path length to most destinations:
• Assume complete shift (easy to model)
• Accurate option
– Extend model to more than one AS away
– Add columns in traffic matrix to designate extra
traffic in case of other network failures
RIPE 52
18
Work in Progress
• Evaluating goodness of models
– Compare actual failures to models
• Evaluating goodness of failover estimates
– Work with both sides of a peering arrangement,
compare failover estimates to simulations
– Compare estimated failover matrices to actual
failures
• Streamlining sharing of information
• Looking for more participants
Contact me to participate in the above
RIPE 52
19
Summary
• Peering/transit links are some of the most
expensive and difficult to provision links
• We can improve capacity planning on such
links by modeling the network
• BGP modeling can be much more complex
than IGP modeling
– Some required information is not even available
• Failover Matrices provide a simple way to
share information without giving away details
• Failover Matrices can be estimated using one’s
own network details
RIPE 52
20
Acknowledgments
• Jon Aufderheide (Global Crossing)
• Clarence Filsfils (Cisco)
RIPE 52
21
References
[1] APRICOT 2005 tutorial: Best Practices for Determining the Traffic
Matrix in IP Networks
[2] APRICOT 2004 tutorial: Traffic Engineering Beyond MPLS
[3] “Modeling the routing of an Autonomous System with C-BGP,” B.
Quoitin and S. Uhlig, IEEE Network, Vol 19(6), November 2005.
[4] “Network-wide BGP route prediction for traffic engineering,” N.
Feamster and J. Rexford, in Proc. Workshop on Scalability and Traffic
Control in IP Networks, SPIE ITCOM Conference, August 2002.
Tutorials [1] and [2] are available at:
http://www.cariden.com/technologies/papers.html
RIPE 52
22