Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Router Design: Part I • Overview of Generic Router Architecture • Input-Queued Switches (Routers) • IP Address Look-up Algorithms • Packet Classification Algorithms Readings: Do required readings (you can skip the math in Section V and Appendix in [Mc+99]); Also do some of the optional readings if interested Q: Any volunteers for scribes? CSci5221: Router Design 1 CSci5221: Router Design . . . . . . Routers in a Network Sample Routers and Switches Cisco 12416 Router up to 160 Gb/s throughput up to 10 Gb/s ports Juniper Networks T640 Router up to 160 Gb/s throughput up to 10 Gb/s ports CSci5221: 3Com 4950 24 port gigabit Ethernet switch Router Design 3 High Capacity Router • Cisco CRS-1 – up to 46 Tb/s thruput • two rack types • line card rack – 640 Gb/s thruput – up to 16 line cards • up to 40 Gb/s each – up to 72 racks • switch rack – central switch stage – up to 8 racks • in-service scaling CSci5221: Router Design 4 Components of a Basic Router • Input/Output Interfaces (II, OI) II IPP OPP . . . • Input Port Processor (IPP) OI output queue routing table . . . – convert between optical signals and electronic signals – extract timing from received signals – encode (decode) data for transmission CP – synchronize signals – determine required OI or OIs from routing table • Output Port Processor (OPP) – queue outgoing cells • shared bus interconnects IPPs and OPPs CSci5221: Router Design Control Processor (CP) » configures routing tables » coordinates end-to-end channel setup together with neighboring routers Generic Router Architecture Data Hdr Header Processing Lookup IP Address Update Header 1 1 Buffer Memory Address Table Data Hdr Header Processing Lookup IP Address Update Header 2 2 Header Processing Lookup IP Address Update Header Address Table CSci5221: Queue Packet Buffer Memory Address Table Data Hdr Queue Packet Router Design N N Queue Packet Buffer Memory 6 Switch Fabric: Three Design Approaches CSci5221: Router Design 7 Switch Fabric: First Generation Routers • Traditional computers with switching under direct control of the CPU • Packet copied to the system’s memory • Speed limited by the memory bandwidth (two bus crossings per packet) Input Port Memory Output Port System Bus CSci5221: Router Design 8 Shared Memory (1st Generation) Shared Backplane CPU Route Table Buffer Memory Line Interface Line Interface Line Interface MAC MAC MAC Typically < 0.5Gbps aggregate capacity Limited by rate of shared memory CSci5221: Router Design 9 Switch Fabric: Switching Via a Bus • Packet from input port memory to output port memory via a shared bus • Bus contention: switching speed limited by bus bandwidth • 1 Gbps bus, Cisco 1900: sufficient speed for access and enterprise routers (not regional or backbone) CSci5221: Router Design 10 Shared Bus (2nd Generation) CPU Typically < 5Gb/s aggregate capacity; Limited by shared bus Router Design Buffer Memory Line Card Line Card Line Card Buffer Memory Buffer Memory Buffer Memory Fwding Cache Fwding Cache Fwding Cache MAC MAC MAC CSci5221: Route Table 11 Switch Fabric: Interconnection Network • Banyan networks, other interconnection nets initially created for multiprocessors • Advanced design: fragmenting packet into fixed length cells to send through the fabric • Cisco 12000: switches Gbps through the interconnection network CSci5221: Router Design 12 Point-to-Point Switch (3rd Generation) Switched Backplane Line Card CPU Card Line Card Local Buffer Memory Routing Table Local Buffer Memory Fwding Table Fwding Table MAC MAC Typically < 50Gbps aggregate capacity CSci5221: Router Design 13 Buffer Placement: Output Port Queuing • Buffering when the aggregate arrival rate exceeds the output line speed • Memory must operate at very high speed CSci5221: Router Design 14 Simple model of output queued switch Link 1, ingress Link rate, R Link 2, ingress R Link 3, ingress R Link 4, ingress R CSci5221: Router Design Link 1, egress Link rate, R Link 2, egress R Link 3, egress R Link 4, egress R 15 Characteristics of an output queued (OQ) switch • arriving packets immediately written into output queue, without intermediate buffering • flow of packets to one output does not affect flow to another output • OQ switch is work conserving: output line always busy when there is a packet in switch for it • OQ switch has highest throughput, lowest average delay CSci5221: Router Design 16 Switching Speed-up Needed Data Hdr Header Processing Lookup IP Address Update Header 1 1 Buffer Memory Address Table Data Hdr Header Processing Lookup IP Address Queue Packet Update Header 2 2 NQueue times line Packet rate Buffer Memory Address Table N times line rate Data Hdr Header Processing Lookup IP Address Update Header Address Table CSci5221: Router Design N N Queue Packet Buffer Memory 17 Buffer Placement: Input Port Queuing • Fabric slower than input ports combined – So, queuing may occur at input queues • Head-of-the-Line (HOL) blocking – Queued packet at the front of the queue prevents others in queue from moving forward CSci5221: Router Design 18 Simple model of input queued switch Link 1, ingress R Link 2 Link 1 R1 Link 2, ingress R Link 3 Link 4 Link 3, ingress R Link 4, ingress R CSci5221: Router Design Link 1, egress R Link 2, egress R Link 3, egress R Link 4, egress R 19 Head-of-line Blocking • Packet at the head of an input queue cannot be transferred, thus blocking the following packets (or cells – packets of fixed size) Cannot be transferred because is blocked by red packet Input 1 Output 1 Input 2 Output 2 Input 3 CSci5221: Cannot be transferred because output buffer full Router Design Output 3 20 Characteristics of an input queued (IQ) switch • arriving packets written into input queue • only one packet can be sent to output link at a time • head-of-line blocking • IQ switch cannot keep output links fully utilized CSci5221: Router Design 21 Buffer Placement: Design Trade-offs • Output queues – Pro: work-conserving, so maximizes throughput – Con: memory must operate at speed N*R • Input queues – Pro: memory can operate at speed R – Con: head-of-line blocking for access to output • Work-conserving: output line is always busy when there is a packet in the switch for it • Head-of-line blocking: head packet in a FIFO cannot be transmitted, forcing others to wait CSci5221: Router Design 22 What is capacity of IQ: Model [optional: Karol et al Globecom’86] •Large input-queued switch with – single FIFO at each input – packet destinations i.i.d. (independently, identically distributed), uniform across outputs – HoL blocked packets not flushed •throughput analysis – – – – – – saturated switch (i.e., always arrival at each input queue) ball/urns model: N balls, N urns focus on first urn Xt - number of balls in urn at time t Dt- number balls removed from all ums at end of time t Dt is switch thruput CSci5221: Router Design 23 Model (cont’d) • At+1 - no. balls dropped into urn 1 at t+1 • Xt+1 = (Xt-1)+ + At+1 • where Dt k Dt k P At 1 k 1 / N 1 1 / N k • E(Dt) = ρN where ρ is output throughput • for large N, binomial distribution can be approximated by Poisson distribution, P( At k ) CSci5221: Router Design k k! e 24 Model (cont’d) E ( A ) EA 2( EA) EX 2(1 EA) 2 2 where EA = ρ, E(A2) = ρ + ρ2 therefore 2 2 EX 2(1 ) EX = 1, therefore 2 2 1 2(1 ) and ρ =2-√2 » 58.6% CSci5221: Router Design 25 A Router with Input Queues Head of Line Blocking Delay The best that any queueing system can achieve. 0% 20% 40% 60% Load CSci5221: Router Design 80% 100% 2 2 58% 26 Solution to Avoid Head-of-line Blocking • How to improve capacity without increasing switching fabric speed ? • Maintain at each input N virtual queues, i.e., one per output – use non-FIFO scheduler, matching input/output Input 1 Output 1 Output 2 Input 2 Output 3 Input 3 CSci5221: Router Design 27 Virtual Output Queueing • assume fixed length packets 1 • each input manages separate queue per output • at each time, matching scheduler finds best possible packets from inputs to said to outputs • maximum-weight matching N CSci5221: Router Design 1 . . . matching scheduler N . . . 28 Matching • Lij(t): no. of packets at input i for output j at t • bipartite graph (V1 x V2,E), E ÍV1xV2 – V1,V2 inputs, outputs; (i,j) ÎE iff Lij(t) > 0 • matching: subset of E such that no two edges are adjacent • maximal matching: no more edges can be added An aside: stability (of a [queueing] system): input output • Assuming the arrival rate is (i.e., # of arrivals per unit of time) less or equal to the system capacity • The system is stable if and only if no queue grows infinitely (under any arrival patterns) as t ® ¥ CSci5221: Router Design 29 Matching problems • maximum size matching – matching with largest number of edges – when traffic uniform, provides 100% utilization – network flow problem, O(N5/2) • maximum weight matching – add weight wij to edge from i to j – – – – • e.g., wij: # of packets from input i to output j in the queue matching with highest weight when wij = Lij(t) provides 100% utilization equivalent to a network flow problem, O(N3) MWM algorithms involve backtracking: i.e. edges laid down in one iteration may be removed later algorithm not amenable to pipelining CSci5221: Router Design 30 Scheduling Algorithms 19 3 4 1 21 18 7 19 19 1 7 Practical Maximal Matchings Max Size Matching Not stable CSci5221: 18 Not stable Router Design Max Wt Matching Stable 31 Switch Algorithms 19 19 1 18 7 Maximal matching Max Size Matching Not stable Not stable Max Wt Matching Stable, low backlogs Better performance Easier to implement CSci5221: Router Design 32 Better Matching Algorithms • Need simple algorithms that perform well • Randomized algorithms with linear complexity available – – – efficient packet processing packets at line speeds high throughput low latencies/backlogs – Tassiulas’ Randomized Algorithm – LAURA – SERENA Use both randomization, history, problem structure and arrival information For more details, see optional reading [SGP02]: “Efficient Randomized Algorithms for Input-Queued Switch Scheduling” by Shah, Giaccone and Prabhakar, IEEE Micro Vol 22, Issue 1, Jan 2002 CSci5221: Router Design 33 Combined Input-Output Queued (CIOQ) Routers • Both input and output interfaces store packets • Advantages input interface output interface – Easy to built • Utilization 1 can be achieved with limited input/output speedup (<= 2) Backplane • Disadvantages – Harder to design algorithms • Two congestion points • Need to design flow control CSci5221: Router Design RO C 34 Output Queue Emulation using CIOQ (with Speed-up) Stable Marriage Problem -- Gale Shapely Algorithm (GSA) • As long as there is a free man m – m proposes to highest ranked women w in his list he hasn’t proposed yet – If w is free, m an w are engaged – If w is engaged to m’ and w prefers m to m’, w releases m’ • Otherwise m remains free • A stable matching exists for every set of preference lists • Complexity: worst-case O(N2) CSci5221: Router Design 35 Stable Marriage Problem • Consider N women and N men • Each woman/man ranks each man/woman in the order of their preferences • Stable matching, a matching with no blocking pairs • Blocking pair; let p(i) denote the pair of i – There are matched pairs (k, p(k)) and (j, p(j)) such that k prefers p(j) to p(k), and p(j) prefers k to j CSci5221: Router Design 36 Example men 1 2 3 4 pref. list 2 4 3 1 1 4 3 2 4 3 2 1 1 2 4 3 women 1 2 3 4 pref. list 1 4 3 2 3 1 4 2 1 2 3 4 2 1 4 3 • If men propose to women, the stable matching is – – – – 1st round: (1,2), (2,1), (3,4), (4,1) -> w1 releases m2 2nd round: (2,4) ->w4 releases m3; 3rd round: (3,3); final match: (1,2), (2,4), (3,3), (4,1) • What is the stable matching if women propose to men? CSci5221: Router Design 37 OQ Emulation with a Speedup of 2 • Each input and output maintains a preference list • Input preference list: list of cells at that input ordered in the inverse order of their arrival • Output preference list: list of all input cells to be forwarded to that output ordered by the times they would be served in an Output Queueing schedule • Use GSA to match inputs to outputs – Outputs initiate the matching • Can emulate all work-conserving schedulers For more info, see the optional reading [C+99] “Matching Output Queueing with a Combined Input Output Queued Switch.” CSci5221: Router Design 38 Line Cards • Interfacing to/from link – – – – – – – Packet forwarding (FIB) Packet filtering (ACLs) Buffer management Link scheduling Rate-limiting Packet marking Measurement CSci5221: Router Design FIB Transmit • Packet handling Receive – Physical link – Switching fabric to/from switch 39 Line Card: Abstract view Header Processing Data Hdr Lookup Update IP Address Header IP Address Hdr Next Hop Address Table CSci5221: Queue Packet Data Router Design Buffer Memory 40 Line Cards: Longest-Prefix Match Forwarding • Forwarding Information Base in IP routers – Maps each IP prefix to next-hop link(s) • Destination-based forwarding – Packet has a destination address – Router identifies longest-matching prefix – Pushing complexity into forwarding decisions FIB 4.0.0.0/8 4.83.128.0/17 12.0.0.0/8 12.34.158.0/24 126.255.103.0/24 destination 12.34.158.5 CSci5221: Router Design outgoing link Serial0/0.1 41 Line Cards: Packet Forwarding Evolution • Software on the router CPU – Central processor makes forwarding decision – Not scalable to large aggregate throughput • Route cache on the line card – Maintain a small FIB cache on each line card – Store (destination, output link) mappings – Cache misses handled by the router CPU • Full FIB on each line card – Store the entire FIB on each line card – Apply dedicated hardware for longest-prefix match CSci5221: Router Design 42 Line Cards: Packet Filtering With Access Control Lists Should arriving packet be allowed in? Departing packet let out? • “Five tuple” for access control lists (ACLs) – Source and destination IP addresses – TCP/UDP source and destination ports – Protocol (e.g., UDP vs. TCP) CSci5221: Router Design 43 ACL Examples • Filter packets based on source address – Customer access link to the service provider – Source address should fall in customer prefix • Filter packets based on port number – Block traffic for unwanted applications – Known security vulnerabilities, peer-to-peer, … • Block pairs of hosts from communicating – Protect access to special servers – E.g., block the dorms from the grading server CSci5221: Router Design 44 Line Cards: Mapping Traffic to Classes • Gold traffic – All traffic to/from President’s IP address – All traffic to/from the port number for DNS • Silver traffic – All traffic to/from academic and administrative buildings • Bronze traffic – All traffic on the public wireless network • Then, schedule resources accordingly – 50% for gold, 30% for silver, and 20% for bronze CSci5221: Router Design 45 A Case Study [Partridge et al ’98] • Goal: show that routers can keep pace with improvements of transmission link bandwidths • Architecture – A CIOQ router – 15 (input/output) line cards: C = 2.4 Gbps (3.3 Gpps including packet headers) • Each input card can handle up to 16 (input/output) interfaces • Separate forward engines (FEs) to perform routing – Backplane: Point-to-point (switched) bus, capacity B = 50 Gbps (32 MPPS) • B/C = 50/2.4 = 20 CSci5221: Router Design 46 Router Architecture packet header CSci5221: Router Design 47 Router Architecture input interface output interfaces 1 Data in 15 Backplane Update routing tables forward engines Control data (e.g., routing) CSci5221: Router Design Data out Set scheduling (QoS) state Network processor 48 Router Architecture: Data Plane • Line cards – Input processing: can handle input links up to 2.4 Gbps – Output processing: use a 52 MHz FPGA; implements QoS • Forward engine: – 415-MHz DEC Alpha 21164 processor, three level cache to store recent routes • Up to 12,000 routes in second level cache (96 kB); ~ 95% hit rate • Entire routing table in tertiary cache (16 MB divided in two banks) CSci5221: Router Design 49 Router Architecture: Control Plane • Network processor: 233-MHz 21064 Alpha running NetBSD 1.1 – Update routing – Manage link status – Implement reservation • Backplane Allocator: implemented by an FPGA – Schedule transfers between input/output interfaces CSci5221: Router Design 50