Download Communication - Princeton University

Internet Topology
COS 461: Computer Networks
Spring 2006 (MW 1:30-2:50 in Friend 109)
Jennifer Rexford
Teaching Assistant: Mike Wawrzoniak
http://www.cs.princeton.edu/courses/archive/spring06/cos461/
1
Returning the Midterm Exam
• Exam scoring break down
–Range: 70-100
–Average: 89
–Median: 92
• See the course Web site
–Exam
–Answer key
2
Goals of Today’s Lecture
• Internet’s two-tiered topology
– Autonomous Systems, and connections between them
– Routers, and the links between them
• AS-level topology
– Autonomous System (AS) numbers
– Business relationships between ASes
• Router-level topology
– Points of Presence (PoPs)
– Backbone and enterprise network topologies
• Inferring network topologies
– By measuring paths from many vantage points
3
Internet Routing Architecture
• Divided into Autonomous Systems
– Distinct regions of administrative control
– Routers/links managed by a single “institution”
– Service provider, company, university, …
• Hierarchy of Autonomous Systems
– Large, tier-1 provider with a nationwide backbone
– Medium-sized regional provider with smaller backbone
– Small network run by a single company or university
• Interaction between Autonomous Systems
– Internal topology is not shared between ASes
– … but, neighboring ASes interact to coordinate routing
4
Autonomous System Numbers
AS Numbers are 16 bit values.
Currently just over 20,000 in use.
•
•
•
•
•
•
•
•
•
Level 3: 1
MIT: 3
Harvard: 11
Yale: 29
Princeton: 88
AT&T: 7018, 6341, 5074, …
UUNET: 701, 702, 284, 12199, …
Sprint: 1239, 1240, 6211, 6242, …
…
5
AS Topology
• Node: Autonomous System
• Edge: Two ASes that connect to each other
4
3
5
2
7
6
1
6
What is an Edge, Really?
• Edge in the AS graph
– At least one connection between two ASes
– Some destinations reached from one AS via the other
d
d
AS 1
AS 1
Exchange Point
AS 2
AS 2
AS 3
7
Interdomain Paths
Path: 6, 5, 4, 3, 2, 1
4
3
5
2
7
1
6
Web server
Client
8
Business Relationships
• Neighboring ASes have business contracts
–How much traffic to carry
–Which destinations to reach
–How much money to pay
• Common business relationships
–Customer-provider
 E.g., Princeton is a customer of AT&T
 E.g., MIT is a customer of Level 3
–Peer-peer
 E.g., Princeton is a peer of Patriot Media
 E.g., AT&T is a peer of Sprint
9
Customer-Provider Relationship
• Customer needs to be reachable from everyone
– Provider tells all neighbors how to reach the customer
• Customer does not want to provide transit service
– Customer does not let its providers route through it
Traffic to the customer
Traffic from the customer
d
provider
advertisements
provider
traffic
customer
d
customer
10
Peer-Peer Relationship
• Peers exchange traffic between customers
– AS exports only customer routes to a peer
– AS exports a peer’s routes only to its customers
– Often the relationship is settlement-free (i.e., no $$$)
Traffic to/from the peer and its customers
advertisements
peer
d
traffic
peer
11
Princeton Example
• Internet: customer of AT&T and USLEC
• Research universities/labs: customer of Internet2
• Local residences: peer with Patriot Media
• Local non-profits: provider for several non-profits
AT&T
USLEC
Internet2
peer
Patriot
12
AS Structure: Tier-1 Providers
• Tier-1 provider
– Has no upstream provider of its own
– Typically has a national or international backbone
– UUNET, Sprint, AT&T, Level 3, …
• Top of the Internet hierarchy of 12-20 ASes
– Full peer-peer connections between tier-1 providers
13
Efficient Early-Exit Routing
• Diverse peering locations
Customer B
– Both costs, and middle
• Comparable capacity at all
peering points
Provider B
– Can handle even load
• Consistent routes
multiple
peering
points
Early-exit
routing
– Same destinations advertised
at all points
– Same AS path length for a
destination at all points
Provider A
Customer A
14
AS Structure: Other ASes
• Tier-2 providers
– Provide transit service to downstream customers
– … but, need at least one provider of their own
– Typically have national or regional scope
– E.g., Minnesota Regional Network
– Includes a few thousand of the ASes
• Stub ASes
– Do not provide transit service to others
– Connect to one or more upstream providers
– Includes vast majority (e.g., 85-90%) of the ASes
15
Characteristics of the AS Graph
• AS graph structure
– High variability in node degree (“power law”)
– A few very highly-connected ASes
– Many ASes have only a few connections
CCDF
1
All ASes have 1 or more neighbors
0.1
0.01
Very few have degree >= 100
0.001
1
10
100
1000
AS degree
16
Characteristics of AS Paths
• AS path may be longer than shortest AS path
• Router path may be longer than shortest path
2 AS hops,
8 router hops
d
s
3 AS hops, 7 router hops
17
Intra-AS Topology
• Node: router
• Edge: link
18
Hub-and-Spoke Topology
• Single hub node
–Common in enterprise networks
–Main location and satellite sites
–Simple design and trivial routing
• Problems
–Single point of failure
–Bandwidth limitations
–High delay between sites
–Costs to backhaul to hub
19
Princeton Example
• Hub-and-spoke
–Four hub routers and many spokes
• Hub routers
–Outside world (e.g., AT&T, USLEC, …)
–Dorms
–Academic and administrative buildings
–Servers
20
Simple Alternatives to Hub-and-Spoke
• Dual hub-and-spoke
– Higher reliability
– Higher cost
– Good building block
• Levels of hierarchy
– Reduce backhaul cost
– Aggregate the bandwidth
– Shorter site-to-site delay
…
21
Backbone Networks
• Backbone networks
–Multiple Points-of-Presence (PoPs)
–Lots of communication between PoPs
–Accommodate traffic demands and limit delay
22
Abilene Internet2 Backbone
23
Points-of-Presence (PoPs)
• Inter-PoP links
–Long distances
–High bandwidth
Inter-PoP
Intra-PoP
• Intra-PoP links
–Short cables between
racks or floors
–Aggregated bandwidth
• Links to other networks
Other networks
–Wide range of media and
bandwidth
24
Where to Locate Nodes and Links
• Placing Points-of-Presence (PoPs)
–Large population of potential customers
–Other providers or exchange points
–Cost and availability of real-estate
–Mostly in major metropolitan areas
• Placing links between PoPs
–Already fiber in the ground
–Needed to limit propagation delay
–Needed to handle the traffic load
25
Customer Connecting to a Provider
Provider
1 access link
Provider
2 access routers
Provider
2 access links
Provider
2 access PoPs
26
Multi-Homing: Two or More Providers
• Motivations for multi-homing
–Extra reliability, survive single ISP failure
–Financial leverage through competition
–Better performance by selecting better path
–Gaming the 95th-percentile billing model
Provider 1
Provider 2
27
Shared Risks
• Co-location facilities (“co-lo hotels”)
– Places ISPs meet to connect to each other
– … and co-locate their routers, and share space & power
– E.g., 32 Avenue of the Americas in NYC
• Shared links
– Fiber is sometimes leased by one institution to another
– Multiple fibers run through the same conduits
– … and run through the same tunnels, bridges, etc.
• Difficult to identify and accounts for these risks
– Not visible in network-layer measurements
– E.g., traceroute does not tell you links in the same ditch
28
Learning the Internet Topology
• Internet does not have any central management
– No public record of the AS-level topology
– No public record of the intra-AS topologies
• Some public topologies are available
– Maps on public Web sites
– E.g., Abilene Internet2 backbone
• Otherwise, you have to infer the topology
– Measure many paths from many vantage points
– Extract the nodes and edges from the paths
– Infer the relationships between neighboring ASes
29
Inferring an Intra-AS Topology
• Run traceroute from many vantage points
– Learn the paths running through an AS
– Extract the hops within the AS of interest
1 169.229.62.1
inr-daedalus-0.CS.Berkeley.EDU
2 169.229.59.225
soda-cr-1-1-soda-br-6-2
3 128.32.255.169
vlan242.inr-202-doecev.Berkeley.EDU
4 128.32.0.249
gigE6-0-0.inr-666-doecev.Berkeley.EDU
5 128.32.0.66
qsv-juniper--ucb-gw.calren2.net
6 209.247.159.109 POS1-0.hsipaccess1.SanJose1.Level3.net
AOL
7 209.247.9.170
pos8-0.hsa2.Atlanta2.Level3.net
8 66.185.138.33
pop2-atm-P0-2.atdn.net
9 66.185.142.97
Pop1-atl-P3-0.atdn.net
10 66.185.136.17
pop1-atl-P4-0.atdn.net
11 64.236.16.52
www4.cnn.com
30
Challenges of Intra-AS Mapping
• Firewalls at the network edge
– Cannot typically map inside another stub AS
– … because the probe packets will be blocked by firewall
– So, typically used only to study service providers
• Identifying the hops within a particular AS
– Relies on addressing and DNS naming conventions
– Difficult to identify the boundaries between ASes
• Seeing enough of the edges
– Need to measure from a large number of vantage points
– And, hope that the topology and routing doesn’t change
31
Inferring the AS-Level Topology
• Collect AS paths from many vantage points
– Learn a large number of AS paths
– Extract the nodes and the edges from the path
• Example: AS path “1 7018 88” implies
– Nodes: 1, 7018, and 88
– Edges: (1, 7018) and (7018, 88)
• Ways to collect AS paths from many places
– Mapping traceroute data to the AS level
– Measurements of the interdomain routing protocol
32
Map Traceroute Hops to ASes
Traceroute output: (hop number, IP)
1 169.229.62.1
AS25
2 169.229.59.225 AS25
Berkeley
3 128.32.255.169 AS25
4 128.32.0.249
AS25
5 128.32.0.66
AS11423 Calren
6 209.247.159.109 AS3356
7 *
AS3356
8 64.159.1.46
AS3356
9 209.247.9.170
AS3356
10 66.185.138.33
AS1668
11 *
AS1668
12 66.185.136.17
AS1668
13 64.236.16.52
AS5662 CNN
Level3
AOL
33
Challenges of Inter-AS Mapping
• Mapping traceroute hops to ASes is hard
– Need an accurate registry of IP address ownership
– Whois data are notoriously out of date
• Collecting diverse interdomain data is hard
– Public repositories like RouteViews and RIPE-RIS
– Covers hundreds to thousands of vantage points
– Especially hard to see peer-peer edges
Sprint
AT&T
d1
Harvard
???
Harvard
B-school
d2
34
Inferring AS Relationships
• Key idea
– The business relationships determine the routing policies
– The routing policies determine the paths that are chosen
– So, look at the chosen paths and infer the policies
• Example: AS path “1 7018 88” implies
– AS 7018 allows AS 1 to reach AS 88
– AT&T allows Level 3 to reach Princeton
– Each “triple” tells something about transit service
• Collect and analyze AS path data
– Identify which ASes can transit through the other
– … and which other ASes they are able to reach this way
35
Paths You Should Never See (“Invalid”)
Customer-provider
Peer-peer
two peer edges
transit through a customer
36
Challenges of Relationship Inference
• Incomplete measurement data
– Hard to get a complete view of the AS graph
– Especially hard to see peer-peer edges low in hierarchy
• Real relationships are sometime more complex
– Peer is one part of the world, customer in another
– Other kinds of relationships (e.g., backup and sibling)
– Special relationships for certain destination prefixes
• Still, inference work has proven very useful
– Qualitative view of Internet topology and relationships
37
Conclusions
• Two-tiered Internet topology
–AS-level topology
–Intra-AS topology
• Inferring network topologies
–By measuring paths from many vantage points
• Next class
–Vivek Pai guest lecture
 See reading assignment on the course Web site
–Mike Wawrzoniak talking about assignment #2
 Start the assignment so you can ask questions
• Next week
–Intradomain and interdomain routing
38