Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Computer network wikipedia , lookup
Recursive InterNetwork Architecture (RINA) wikipedia , lookup
Cracking of wireless networks wikipedia , lookup
Piggybacking (Internet access) wikipedia , lookup
Net neutrality wikipedia , lookup
Airborne Networking wikipedia , lookup
Routing in delay-tolerant networking wikipedia , lookup
Mapping the Internet Topology Traceroute A Basic Tool for Topology p gy Discovery y Yuval Shavitt School of Electrical Engineering http://www.eng.tau.ac.il/~shavitt http://www.netDimes.org Simple delay/loss probing with ping ICMP C:\>ping www.fer.hr ICMP is the IP error diagnosis protocol. Pinging www.fer.hr [161.53.72.111] with 32 bytes of data: IP header Type Reply from 161.53.72.111: bytes bytes=32 32 time time=113ms 113ms TTL=49 TTL 49 Reply from 161.53.72.111: bytes=32 time=111ms TTL=49 Reply from 161.53.72.111: bytes=32 time=113ms TTL=49 Reply from 161.53.72.111: bytes=32 time=118ms TTL=49 Code Checksum Ping statistics for 161.53.72.111: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 111ms, Maximum = 118ms, Average = 113ms Sequence number Any ICMP data ICMP Message Types ping • Can be used to – Check host is alive – Measure RTD to a server • Application layer “ping” – One can generate application layer messages to test application reaction time – Most common: • TCP SYN message to port 80 1 Type No. Meaning 0 Echo reply 3 Destination unreachable 4 Source quench 5 Redirect 8 Echo 9 R t advertisement Router d ti t 10 Router solicitation 11 Time exceeded 12 Parameter problem 13 Timestamp 14 Timestamp reply 15 Information requeste 16 Information reply PING IP datagram format IP protocol version number header length (bytes) “type” of data 32 bits length ver len service fragment 16-bit identifier flgs offset time to upper Internet layer live checksum max number remaining hops (decremented at each router) traceroute total datagram length (bytes) head. type of for fragmentation/ reassembly • Useful to learn the route characteristics between two hosts. • Sends a series of probes to successive nodes along a route to an intended destination and records the source address and time delay of the message returned by each. • Based on ICMP “TTL expired” message 32 bit bi source IP address dd 32 bit destination IP address upper layer protocol to deliver payload to E.g. timestamp, record route taken, pecify list of routers to visit. Options (if any) data (variable length, typically a TCP or UDP segment) ICMP Message Types traceroute time A B Regular UDP packets • successive TTLs ICMP “TTL expired” message ICMP “port unreachable” message C D E Type No. Meaning 0 Echo reply 3 Destination unreachable 4 Source quench 5 Redirect 8 Echo 9 R t advertisement Router d ti t 10 Router solicitation 11 Time exceeded 12 Parameter problem 13 Timestamp 14 Timestamp reply 15 Information requeste 16 Information reply Tracing route to www.fer.hr [161.53.72.111] over a maximum of 30 hops: <1 19 17 21 19 20 69 82 101 105 117 113 120 114 120 114 ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms Trace complete. <1 20 22 19 23 20 69 82 98 105 112 115 122 112 119 114 2 ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms <1 19 20 19 18 20 69 82 98 105 113 115 123 119 119 113 ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms 192.168.200.254 vxr.tau.ac.il [132.66.8.10] c6509.tau.ac.il [132.66.8.20] tel-aviv.tau.ac.il [132.66.4.1] gp1-tau-fe.ilan.net.il [128.139.191.70] iucc.il1.il.geant.net [62.40.103.69] i i il.it1.it.geant.net i [[62.40.96.154]] it.ch1.ch.geant.net [62.40.96.33] ch.at1.at.geant.net [62.40.96.1] at.hu1.hu.geant.net [62.40.96.178] hu.hr1.hr.geant.net [62.40.96.145] carnet-gw.hr1.hr.geant.net [62.40.103.218] 193.198.228.6 193.198.229.10 161.53.16.14 duality.cc.fer.hr [161.53.72.111] Code 0 1 2 3 6 7 description dest. network unreachable dest host unreachable dest protocol unreachable dest port unreachable dest network unknown dest host unknown traceroute traceroute versions C:₩>tracert www.fer.hr 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Type 3 3 3 3 3 3 • UNIX: – default send UDP packets • Start at port 33435, and increment port per packet! Whyy thiss iss not o advisable? dv s b e? • W – traceroute –l sends ICMP “ECHO request” – tcptraceroute uses TCP SYN messages • If port is close gets RST reply • If port is open gets SYN ACK and reply with RST • Best to overcome firewalls • Windows – ICMP “ECHO request” From Traceroute to IP Maps Alias Resolution Ping (echo request) X • Problem 1: or UDP X (high port number – TR gives interface IP not router ID • Problem 2: M – TR can result in false links • Problem 3: echo reply Y – Non responding routers ICMP (port unreachable) Y ⇒ IP X = IP Y Traceroute and Load-Balancing Actual topology: Src A C TTL = 2 L B E Dst D TTL = 3 Inferred topology: A Src C False link E L B Aliasing Based on IP Identifier • x<y<z, z-x small Æ likely aliases • If |x-y|>200 Æ Aliases are disqualified, third packet is not sent Dst D Missing nodes and links 1 6 Anomalies: false loops and cycles Load Balancing can lead to ‘Loops’ TTL = 3 Actual topology: Src A D L B Dst C TTL = 2 TTL = 4 TTL = 2 Inferred topology: TTL = 4 TTL = 3 Src L D B real topology traceroute possible outcome 17 3 Dst Reducing The Load Balancing Problem Anomalies: false diamonds A Actual topology: • Load balancing can be done in two ways: Src Dst E L 1. Packet based 2. Flow based (S IP, D IP, S port, D port, prot) • Sending all probes as a single flow will force the same route selection in flow based Load balancer • How to do this? C B D A C B D Inferred topology: Src Dst E L 19 Measurement artifacts are common From LIP6 vantage point: • Diamonds appear in 30% of the destinations Paris traceroute removes 10,662 from 19,159 (56%) • Loops appear in 4.5% of the measured routes Paris traceroute removes 5,047 5 047 from 5,795 5 795 (87%) Paris traceroute • Solves the problem with per-flow load balancing Probes to a destination belong to same flow • How to identify probes? Use the UDP checksum • Does not address per-packet per packet load balancing • Cycles appear in 0.25% of the measured routes Checksum 2 Port 1 Checksum 1 A Port 1 Paris traceroute removes 3,886 from 5,674 (68%) • Other causes Routing changes NAT boxes Buggy routers Per-packet load balancing Src TTL = 3 E Dst D TTL = 1 21 Anomalies: Loops caused by NAT boxes Response TTL = 254 IP Identifier = 12375 Src A Response TTL = 252 IP Identifier = 9356 Dst (NAT) Dt Dst B B TTL = 2 TTL = 3 2 Src 4 TTL = 2 L B 22 24 Checksum 3 Port 1 C A Dst C TTL = 3 Anomalies: Loops caused by buggy routers Src A X Rejects the probe with a TTL of 0 and = 1 Forwards theTTL probe sends it back to the with TTL equal to source 0 Response TTL = 253 IP Identifier = 5286 Src 23 B Dst Forwards the probe probe with TTL equal to with a TTL of 0 and 0 sends it back to the source Rejects TTL TTL == 12the B Dst -bash$ traceroute Dst traceroute to Dst 1 B 0.289 ms 2 B 0.278 ms 3 Dst 0.578 ms -bash$ traceroute-paris Dst traceroute to Dst 1 B 0.289 ms !T0 2 B 0.278 ms 3 Dst 0.578 ms Non-Responding Routers True network C:₩>tracert www.colbud.hu 2 Tracing route to www.colbud.hu [81.182.250.153] over a maximum of 30 hops: 11 444 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 33 7 5 66 88 9 11 <1 19 20 21 20 26 91 97 95 96 110 * 112 112 114 120 ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms <1 21 21 20 22 22 92 97 96 96 112 * 110 114 112 122 ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms <1 18 21 19 19 21 92 97 93 150 114 * 111 110 114 124 ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms 192.168.200.254 vxr.tau.ac.il [132.66.8.10] c6509.tau.ac.il [132.66.8.20] tel-aviv.tau.ac.il [132.66.4.1] gp1-tau-fe.ilan.net.il [128.139.191.70] iucc.il1.il.geant.net iucc il1 il geant net [62 [62.40.103.69] 40 103 69] il.nl1.nl.geant.net [62.40.96.117] nl.de1.de.geant.net [62.40.96.101] ffm-b2-pos2-3.telia.net [213.248.77.89] ffm-bb2-pos2-3-0.telia.net [213.248.64.177] bpt-b1-pos2-0.telia.net [213.248.64.26] Request timed out. 10ge-0-0.core0-ip2.net.telekom.hu [145.236.85.2] tenge1-2.core0.adatpark.hu [145.236.89.10] fixip-lns2.adatpark.hu [195.228.253.58] 153-250-182-81.adsl-fixip.axelero.hu [81.182.250.153] Trace complete. 10 10 26 True network revisited Measured network Suppose one router is unresponsive: 1 2 4 111 3 44 4 4 33 7 7 7 5 1 7 66 ?? 8 7 9 11 6 10 10 10 8 28 27 Unresponsive Routers - Example Measured network 1 4 3 4 7 7 10 ? 7 10-?-7 1 10 7 True network with 3 unresponsive routers Measured network with 51 unknown nodes The non-blue nodes are unresponsive. The non-blue nodes are unknowns (gaps). 30 5 6 6-?-7 29 Unifying Unknowns Is it possible to identify a group of unknown nodes that represent the same unresponsive router? AS Topology Discovery 31 The Internet Structure The Internet Structure routers The AS graph from IP to AS routes IP to AS C:₩>tracert www.fer.hr Tracing route to www.fer.hr [161.53.72.111] over a maximum of 30 hops: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 <1 19 17 21 19 20 69 82 101 105 117 113 120 114 120 114 ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms <1 20 22 19 23 20 69 82 98 105 112 115 122 112 119 114 ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms <1 19 20 19 18 20 69 82 98 105 113 115 123 119 119 113 ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms ms 192.168.200.254 vxr.tau.ac.il [132.66.8.10] c6509.tau.ac.il [132.66.8.20] tel-aviv.tau.ac.il [132.66.4.1] gp1-tau-fe.ilan.net.il [128.139.191.70] iucc.il1.il.geant.net [62.40.103.69] i i il.it1.it.geant.net i [[62.40.96.154] ] it.ch1.ch.geant.net [62.40.96.33] ch.at1.at.geant.net [62.40.96.1] at.hu1.hu.geant.net [62.40.96.178] hu.hr1.hr.geant.net [62.40.96.145] carnet-gw.hr1.hr.geant.net [62.40.103.218] 193.198.228.6 HR-ZZ 193.198.229.10 161.53.16.14 duality.cc.fer.hr [161.53.72.111] Trace complete. 378 6 20965 2108 private network Tel Aviv Uni. AS378 ILAN MACHBA DANTE AS20965 GEANT CARnet CARnet AS2108 How map IP to AS? Rifining IP-to-AS Mapping • Start with initial IP-to-AS mapping • Routing address registry – Mapping from BGP tables is usually correct – Good starting point for computing the mapping – Voluntary public registry such as whois.radb.net – Used by prtraceroute and “NANOG traceroute” – Incomplete and quite out-of-date • Collect C ll t many BGP andd traceroute t t paths th – Signaling and forwarding AS path usually match – Good way to identify mistakes in IP-to-AS map • Mergers, acquisitions, delegation to customers • Origin AS in BGP paths • Successively refine the IP-to-AS mapping – Public BGP routing tables such as RouteViews – Incomplete and inaccurate… but usually right – Find add/change/delete that makes big difference – Base these “edits” on operational realities • Multiple Origin ASes (MOAS), no mapping, wrong mapping Extra AS due to Sibling ASes • Sibling: organizations with multiple ASes: – E.g., Sprint AS 1239 and AS 1791 – AS numbers equipment with addresses of another A B H D E A F B G C D C Traceroute AS path Extra AS due to Internet eXchange Points • IXP: shared place where providers meet – E.g., MAE-East, MAE-West, LINX, AMS-IX – Large number of fan-in fan in and fan-out fan out ASes E F A E B F B F G C G C G D A Traceroute AS path BGP AS path Sibling ASes “belong together” as if they were one AS. Interfaces in multiple ASes E BGP AS path Physical topology and BGP session graph do not always match. Weird Paths Due to Unannounced Addresses 12.0.0.0/8 A A-B-C A B C C-B-A A B C B C does not announce part of its address space in BGP (e.g., 12.1.2.0/24) AC A C C Are A and C neighbor ASes? What AS does the middle router belong to, B or C? AC BAC BC Fix the IP-to-AS map to associate 12.1.2.0/24 with C 7 Traceroute Inherent Problems Same problems as with router level maps: • Load balancing • Routing change during traceroute • Effects: Making AS Maps Larger – Wrong links – Loops – Missing links Revealing the Internet Structure Revealing the Internet Structure Revealing the Internet Structure Revealing the Internet Structure Diminishing return! ⇓ Deploying more boxes does not pay-off 7 new links 30 new links NO new links 8 Revealing the Internet Structure Number of VPs To obtain the ‘horizontal’ links we need strong presence in the edge • Large instrumentation boxes (Ark) – Cannot reach more than 10s of VPs • Community base software agents (DIMES) – Reached 100s VP – Can grow much more DIMES • Very light agents (Ono) – Reached 1000s VP – Hard to clean noise • Small hardware devices (RIPE ATLAS) – Reached 100s VP TAU’s DIMES CAIDA’s Archipelago (Ark) • ~1000 active agents daily DIMES • 54 boxes May 2011 May 2011 So how many is enough? RIPE’s ATLAS • 400 active (600 distributed) • [Chen et al. 02], [Bradford et al. 01]: when you combine more and more points of view the return diminishes veryy fast • A few VPs are enough – The mass of the tail is significant contribution No. of views 9 May 2011 Data Set How to Measure Contribution? • Data is obtained from DIMES – Community-based infrastructure, using almost 1000 active measuringg software agents g – Agents follow a script and perform ~2 probes per minute (ICMP/UDP traceroute, ping) – Most agents measure from a single AS (vp) • Graph size is not the only factor. • We want our Internet view to share the same properties as the real Internet? – Remove measurement bias • How to quantify this? • But some (appear to) measure from more… • Data need to be filtered to remove artifacts – Traceroute data collected during March 2008 Measurements Per Agent Filtering the data • For each agent and each week, classify how many networks it measured the Internet from Typical yp cases: – ASi:15300, ASj:8 – ASi:10000, ASj:3178 – ASi:10000, ASj:412 , ASk:201 – 18000, 12, 11, 9, 9, 3, 3, 2, 2, 1, 1, 1, 1, 1, …. Week 4,2008 Agents per Network Measurements per Network 500 10 Diminishing Returns? • Barford et. al. – the utility of adding many vps quickly diminishes – In terms of ASes and AS AS-links links • Shavitt and Shir – utility indeed diminishes but the tail is long and significant Filtering Results • 96% of the agents have less than 4 different vps • High degree ASs tend to have more agents • High number of measurements for all vps degrees – Tail is biased towards horizontal links • We wish to quantify how different aspects of AS-level topology are affected by adding more vps Creating topologies per VP Topology Size • The return (especially for AS links) does not diminishes fast! sort by VP with small local topology can contribute many new links! Convergence of Properties Direction of Detected Links • Taking several common AS-level graph properties, and analyze their convergence as local topologies p g are added • For each link: Plot max adjacent AS degree and max adjacent ASes degree difference – Keeping the sort order by number of links • Slow convergence indicates the need to have broad and diverse set of vps 11 High degree difference – indicates radial links towards the core Low degree g difference – indicates tangential links and links between smallsize ASes Power-law and Max Degree Density and Average Degree Slow convergence of density and average degree – easy to detect ASes but difficult to find all links Fast convergence of maximal degree – core links are easily detects Fair convergence of power-law exponent Revisitng Sampling Bias Betweenness and Clustering • Lakhina et al. – AS degrees inferred from traceroute sampling are biased – ASes in vicinity to vps have higher degrees – Power-law might be an artifact of this! • Dall’asta et al. – no…it is quite possible to have unbiased degrees with traceroutes • Cohen et al. – when exponent is larger than 2, resulting bias is neglible Dataset VPs and Distances Low degree ASes are seen from less vps than high-degree Ases…this makes sense! Fast convergence of max bc – Level3 (AS3356), a tier-1 AS is immediately detected as having max bc Radial links decrease cc Tangential links increase cc Evaluating Sampling Bias • For each AS find: – All the vps that have it in their local topology – The Valley-Free Valley Free distance in hops In our dataset, most ASes have a vp that is only 1-2 hops away! Up-hill to the core (c2p), side-ways in the core (p2p) and down-hill from the core (p2c) 12 Revisiting Diversity Bias • What is the effect of diversity in vps geolocation and network type? – Some infrastructures rely on academic networks for vp distribution – does it have an effect on the resulting topology? Average Distance per Degree Low degree ASes are seen from farther vps sampling bias? vps…sampling • We compare iPlane and DIMES – Classify AS into types: t1,t2, edu, comp, ix, nic using Dimitropoulos et al. In Search of Ground Truth No real bias! More VPs are located in high-degree ASes• There are high-degree ASes that are seen from “far” vps• Broad distribution – all ASes are pretty close-by to a vp!• Diversity Bias Evaluation • One week is not sufficient for active measurements iPlane uses many PlanetLab nodes (edu), while DIMES resides mostly at homes (tier (tier-2) 2) • Both iPlane and DIMES have lower average degrees than RouteViews – Except iPlane’s edu and ix! – Diversity bias exists – need diverse vp types! Conclusion • VP distribution is important – Number, AS type, geo-location Indeed DIMES have higher t2 and comp degrees and iPlane have higher edu degrees – results are slightly biased to vps’ types! Measuring Within a Network • Comparing vp average degrees to quantify the effect of measuring within a network • AS-level graph properties are affected – Some converge very fast – Other converge slowly • Community based projects have practically unlimited growth potential! 13 Indeed, the average degree when measuring within a network is mostly higher (hmm…tier-1 doesn’t count cause most vps are the same!) BGP Listeners • Passive listening to BGP messages can easily give AS level topology information PoP Level Maps – U. of Oregon: RouteViews – RIPE • Problems: – Aggregation – limited The Internet Structure What the PoP is ? • PoP – Point of Presence of the ISP routers The Internet Structure The AS graph 14 The PoP level graph The Internet Structure The AS graph iPlane PoP Discovery Approach monitor monitor PoP Discovery (RocketFuel) • Group interfaces to routers • use DNS names – S1-bb11-nyc-3-0.sprintlink.net is a Sprint router in New York City • use connectivity information – if a router connects only to routers in Seattle, it is in Seattle Delay measurements • Use the router role in the topology? monitor – only backbone routers connect to other cities – use DNS names • s1-gw2-sea-3-1.sprintlink.net is a Sprint gateway router [Madhyastha et al. IMC 2006] Bad Routing Example Tel Aviv University [Spring et al. SIGCOMM 2002] Problems with this Approch • Delay measurements are noisy – Change of time – Routing anomalies: permanent stretch < 6 km Azriely Center TAU to Azriely Azriely to TAU BEZEK-NET, AS6810 15 Can we measure link delay? PoP Discovery (DIMES) C:₩>tracert www.fer.hr Tracing route to www.fer.hr [161.53.72.111] over a maximum of 30 hops: Link Min. delay 0 1 <1 ms <1 ms <1 ms 192.168.200.254 19 19 2 19 ms 20 ms 19 ms vxr.tau.ac.il [132.66.8.10] -2 17 3 17 ms 22 ms 20 ms c6509.tau.ac.il [132.66.8.20] 2 19 4 21 ms 19 ms 19 ms tel-aviv.tau.ac.il [132.66.4.1] -1 18 5 19 ms 23 ms 18 ms gp1-tau-fe.ilan.net.il [128.139.191.70] 2 20 6 20 ms 20 ms 20 ms iucc.il1.il.geant.net [62.40.103.69] 49 7 69 ms 69 ms 69 ms il.it1.it.geant.net i i i69 [[62.40.96.154]] 13 82 8 82 ms 82 ms 82 ms it.ch1.ch.geant.net [62.40.96.33] 16 98 9 101 ms 98 ms 98 ms ch.at1.at.geant.net [62.40.96.1] 7 105 10 105 ms 105 ms 105 ms at.hu1.hu.geant.net [62.40.96.178] 7 112 11 117 ms 112 ms 113 ms hu.hr1.hr.geant.net [62.40.96.145] 1 113 12 113 ms 115 ms 115 ms carnet-gw.hr1.hr.geant.net [62.40.103.218]7 120 13 120 ms 122 ms 123 ms 193.198.228.6 2 112 14 114 ms 112 ms 119 ms 193.198.229.10 7 119 15 120 ms 119 ms 119 ms 161.53.16.14 -6 113 16 114 ms 114 ms 113 ms duality.cc.fer.hr [161.53.72.111] Negative delays • Most delay measurements are OK • But some have noise • Use structure to clean ‘bad’ measurements Trace complete. What is the graph obtained from running traceroutes thru a POP? A delay of a link inside TAU Link Delay Measurements Histogram 5000 4500 4000 Distribution of the delay among 1 ms bins s 3500 3000 2500 2000 1500 1000 500 0 -150 -100 -50 0 50 Link delay [ms] 100 150 200 250 negative delay Motifs in IP interface Graphs Network Motifs and Significance • Z-Score in IP level Graphs • “Network Motifs: Simple Building Blocks of Complex Networks” R. Milo, S. Shen-Orr, S. Itzkovitz, N. Kashtan, D. Chklovskii, and U. Alon. Network motifs: simple building blocks of complex networks. Science, 2002. • Qualitative measure of statistical significance significance, the Z score: • We should look for motifs with small weights of the edges – representing small physical delays. 16 Z= X −μ σ POP Extraction Flow Example Remove all edges with: •delay higher than 5 ms •small number of measurements. Unify adjacent parent/child groups to POPs cores, then unify adjacent POPs Build an induced sub-graph of G for each connected component. Run Localization Algorithm (on subgraphs) Divide the sub sub-graph graph to parent/child groups (on sub-graphs) Connect unassociated nodes to the nearest POP cores. [Feldman & Shavitt, Globecom ‘08] Localization Algorithm • Checks whether all nodes that belong to the same parent/child group are located at the same physical location (belongs to the same POP). • Are nodes A and B collocated? (multi-dimensions) Parent-Child Classification A I A B II B • If we assume Gaussian noise – in case I nodes are closer • If we assume Impulse noise – most likely we should select case II Localization Algorithm (cont.) • For bipartite graph G(V,E) with weight function • For each pair of parent nodes • Define common children group CC with members {cc1,cc2,…} Parent pair - both of them points to two or more nodes. Child pair - both of them are pointed by two or more nodes. Groups of parent/child nodes are union of parent/child couples. 17 C2 P1 P2 blue nodes - parent , red nodes - child, blue and red nodes - both parent and child, dashed nodes - neither of two. Localization example • Is 5 co-located with 1,2,3? – No: by node 10 – Yes: by nodes 9 & 11 – Maybe: by node 6 • Clearly {6,7,8} are not collocated with {9,10,11} • Error Ratio Vector C1 POP Extraction connect nodes and core elements. • Repeat Localization algorithm for child nodes, too • Unify adjacent parent/child groups to formulate POP cores • Unify adjacent POP cores • Optional: Connect “lonely nodes” (singletons) to POPs. PoP Discovery – – – – Log function brings small errors towards zero. Using log function on x and 1/x has equal impact Median is used to disregard small number of outliers Note: It is also possible to take an absolute difference into account too. However no significant benefit was observed. • Using Traceroute measurements – Increased number of discovered PoPs compared to 1 week period. – Less sensitive to topology p gy changes g than 4 weeks p period. No. of PoPs • Compare | median(log( ER (u , v)) | to a threshold PoP Discovery • Running on bi-weekly basis Time Frames Localization Algorithm (cont.) IPs in PoPs Distinct Edges 1 week to 1 week <1% <1% ±20% 1 week to 2 weeks +58% +79% +43% 2 weeks to 4 weeks +10% +15% +59% PoP Discovery • The number of discovered PoPs directly relates to the number of discovered IP edges. – 30M-40M measurements per week. – 5.5M-6.5M distinct edges discovered. – ~1000 1000 agents in over 200 ASes are used for the measurements. – 2.5M IP addresses in over 26,000 ASes are being targeted. – Using Median algorithm to estimate distance between nodes. PoP Discovery • Discovered PoPs – ~4400 discovered PoPs. – Over 50,000 IPs within discovered PoPs. • Over 100,000 IPs with singletons g • Discovered mostly large PoPs and not access PoPs. • Enhancements – Targeting iPlanes’s PoP’s IP addresses – increased the number of discovered PoPs by less than 20%. – Targeted measurements to specific AS doubled the number of discovered PoPs in small ASes. • Had some effect in large ISPs but not to that extent. 18 The Internet Structure PoP Discovery • Sensitivity to delay threshold: Number of PoP IPs The PoP level graph Number of PoPs • Sensitivity to number of measurements threshold: Number of PoP IPs Number of PoPs The AS graph Embedding PoPs in Geography Evaluating GeoLocation databases GeoLocation: Possible Approaches • Use available GeoIP databases – Better first validate their accuracy • Use triangulation from known locations – We know where PlanetLab nodes are – Can we clean the noise? Verizon/MCI/UUNET (ASN 703) 10-nodes PoP (w/Singletons) Evaluation of Geolocation Databases How to Evaluate GeoIP DBs • Seven databases were used for the evaluation. – – – – – – – NetAcuity (Digital Element) – High end GeoBytes GeoIP (MaxMind) IPligence Max IP2Location HostIP.info – Free service Spotter – Research tool • Dataset: DIMES measurements, March 2010 – 52K IP addresses (+ 52K singletons IP addresses) – 3800 PoPs 19 • Main problem: NO ground truth! • Idea: compare several databases – Problem: some databases may get their data from the same source • Better idea: – Use aggregation into PoPs to check DB coherency Evaluation methods • • • • Vendor Reported Accuracy Null Replies Agreement within a database - coherency “Ground Truth” location Comparison Between databases – Similarity – By majority Vote • Database anomalies †US state accuracy 115 PoP Location Null Replies • For each IP in the PoP (N IPs), each database (M) get a vote on the geo-location – Number of votes N•M • Using the votes we define the PoP location and convergence radius 118 Stage 1 Alternative Method 117 PoP Location and ‘Convergence’ • CAIDA use GeoIP DB voting for IP location: – Select a threshold. They use 80km – Cluster locations within the threshold – Node can belong to different clusters – The largest cluster is selected as winner – Use the centroid of the winning cluster 119 20 Stage 1 PoP spread radius PoP spread radius CDF of Location Votes Percentage Within 500km from PoP Center Correlation among Databases CDF of Range of Convergence within Databases Ground Truth evaluation • Using CAIDA’s 25K “Ground Truth” IP addresses – January-2010 database, based on DNS & ISP collaboration – In the results, city range considered at 100km range Heatmap – Median distance between databases 85 216 IP hits Country Match G b t Geobytes 67 3% 67.3% 80 1% 80.1% HostIP.Info 28.1% 89.0% IP2Location 100% 76.0% IPligence 100% 76% Netacuity 67.9% 96.9% Spotter 54.1% --- City Match 10.1K wrongly 26 5%in 26.5% located Washington DC 17.9% 13.3% 0.7% 20.5K wrongly 79.1% located in Washington 27.8%DC CDF- distances between databases Comparison with CAIDA 85 Database Database Correlation 216 134 134 IPligence- MaxMind (CAIDA) 21 Evaluating GeoLocation databases Database Anomalies - Disagreement Between Databases Evaluating GeoLocation databases DIMES Database Anomalies - Disagreement Between Databases Global Crossing (ASN 3549) 160-nodes PoP (w/Singletons) Verizon/MCI/UUNET (ASN 703) 10-nodes PoP (w/Singletons) Database Anomalies – False Location Replies Improving Active Measurements Spotter ` Geolocation tool, active measurements based ` Our findings (Tech report): ◦ Failed to locate a third of the IP addresses ◦ Its range of convergence was considerably higher than other databases ` Idea: Leverage DIMES infrastructure to improve Spotter ◦ More vantage points, in locations possibly closer to the target ◦ Larger dataset ◦ Conclusions from PoP study help analyze incorrect location assignments ` Goal: study the limits of active measurement based geolocation Evaluating GeoLocation databases Agreement Between Databases – By Majority Vote Qwest as an example • • • • • 70 PoPs were discovered by the algorithm MaxMind assigned the PoPs to 55 different locations HostIP.Info assigned the PoPs to 46 different locations IP2Location assigned the PoPs to 35 different locations IPligence located the PoPs in only one distinct location; – All the PoPs were placed in Denver, where Qwest HQ are located. – Out of 20291 Qwest entries in IPligence, 20252 are located in Denver. • MaxMind had the same problem as IPligence in their May-2009 DB, but it was fixed in July-2009 DB. Agreement Between Databases – By Majority Vote CDF of Database Location Deviation From PoP Median. Long tail. 22 DIMES Summary What Topology Can Tell US DIMES Many bad news: • Ground truth has bias • Coherency ≠ Accuracy – BUT: incoherency ⇒ inaccuracy • Database correlation – Majority vote is tricky Most results appear in arXiv:1005.5674, May 2010 & JSAC Dec 2011 133 Background (cont.) • [Dhamdhere and Dovrolis , CoNext 2010] – a new Internet model that captures the Internet transition from a hierarchyy of transit pproviders to a flatter interconnection of peers. Background • Internet economics is changing rapidly • [Gill, et al. PAM 2008] – Large content providers (CPs) bypass many tier-1 ISPs by pushing their networks closer to the users – The Internet becomes flatter. • [Labovitz et al., SIGCOMM 2010] – Traffic from CPs is increasing in time – Most of it is routed outside the core Methodology • Collect one month of traceroutes every 3 months • IP to AS mapping. pp g AS route to topology p gy – When crossing IXP ignore it A-X-B ⇒ A-B – Mapping error do not change the trends • DIMES has a large vantage point churn – Use iPlane data instead – DIMES shows, in most cases, the same trend but with larger topologies and more noise 23 Our Work • A longitudinal study over 5 years • Observe the change in the Internet ecosystem using topology instead of traffic • Focus on content providers and tier-1 ISPs ASes Measured Degree Content Providers: A clear increase in the degree MSN Google Yahoo! Amazon Facebook Contend Provides Tier-1 ISPs • Google • AT&T • Yahoo! • Level 3 • MSN • Sprint • Facebook • Qwest • Amazon • Global Crossing CPs Neighbor Degree Amazon and Facebook: Move from reliance on tier-1 to more diverse neighbors CP Neighbor Country Tier-1 ISPs: Mostly decreasing Tier-1 Neighbors 2007 2008 2009 2010 April Veteran CPs already exploited their US growth potential 24 New CPs grow in the US Degree increase: •Lose low degree customers •Other O e customers cus o e s increase degree (?) Clustering Coefficient CP Neighbor Type CPs: CC mostly decreasing New CPs diversify their neighbor type Decrease the percentage of LTP k-Pruning Analysis Tier-1: Mostly increasing (losing small stub customers) k-Pruning Analysis k-Pruning Analysis 1-shell 25 1-shell k-Pruning Analysis k-Pruning Analysis 1-shell k-Pruning Analysis 1-shell k-Pruning Analysis 1-shell 1-shell 2-shell 2-shell 3-shell = core k-Pruning Analysis k-Pruning Analysis 1-shell 1-shell 2-shell 2-shell 3-shell 3-shell = core 4-shell = core 26 Core Make-up Shell Index In DIMES: MSN is below Yahoo! and Google Content/Access/ Hosting Provider Large Transit Provider Small Transit Provider Enterprise Customer [Dhamdhere and Dovrolis, IMC’08] Not only LTPs, Not noticed by [Carmi et al., PNAS 2007] Centrality • The veteran CP are in the core (or close) •The new CPs advance very fast towards the core All tier-1 ISPs are in the core How Do CPs Connect? • Betweenness centrality 1. Use shortest paths on the undirected AS graph 2. Examine all traceroute containing the AS • • Calculate the portion of traceroute where the AS is in the middle of the AS level route More accurate than using ToR + valley free SPR – The two methods gave similar results CPs tend to use IXPs • PageRank Plotted with DIMES data CPs Betweenness Centrality Tier-1 Betweenness Centrality Note the scale Google and Yahoo! main AS serve as a transit to sibling ASes: In 14% and 22% of the traces, respectively. 27 AS10310 AS15169 • Sprint & Level3 are slowly decreasing • The rest are stable Summary PageRank Note the scale • CPs increase their connectivity • Diversify their connections: – From mostly tier tier-11 to tier-2 tier 2 and others – From US centric to global • Tier-1 centrality is decreasing • Indeed we see a flatter Internet 28