* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Interconnection networks 1
Survey
Document related concepts
Bus (computing) wikipedia , lookup
Zero-configuration networking wikipedia , lookup
Computer network wikipedia , lookup
Network tap wikipedia , lookup
Asynchronous Transfer Mode wikipedia , lookup
IEEE 802.1aq wikipedia , lookup
Deep packet inspection wikipedia , lookup
Recursive InterNetwork Architecture (RINA) wikipedia , lookup
Cracking of wireless networks wikipedia , lookup
Wake-on-LAN wikipedia , lookup
List of wireless community networks by region wikipedia , lookup
Airborne Networking wikipedia , lookup
Real-Time Messaging Protocol wikipedia , lookup
Transcript
ENGS 116 Lecture 19 1 Interconnection Networks Vincent H. Berk November 14, 2005 Reading for Friday: 8.1-8.7 Reading for Monday: 8.8-8.13 ENGS 116 Lecture 19 2 Project Reports • • • • • • • Due by Beginning of class next Monday, November 24 Content – Introduction and description of the topic – Coverage of topic: breadth/depth, appropriate background information – Analysis and discussion – References: correct citations, proper form Writing – Spelling – Grammar – Style and presentation Assume that the reader is familiar with basic architecture concepts Must use appropriate citations. Argument all your decisions. Email all source code to <[email protected]> ENGS 116 Lecture 19 3 Project Presentations • • • • 15 minutes – absolutely no more – Practice your timing! All group members talk 24th (4) of November, 1st (4) of December Present: – – – – research question approach results conclusions • EVERYONE ATTENDS these presentations ENGS 116 Lecture 19 4 Networks • Common topics of conversation: – direct (point-to-point) vs. indirect (multi-hop) – topology (e.g., bus, ring, directed acyclic graph, star) – routing algorithms – switching (aka multiplexing) – wiring (e.g., choice of media, copper, coax, fiber) • What really matters: – latency – bandwidth – cost – reliability ENGS 116 Lecture 19 5 ABCs of Networks • Starting point: Send bits between 2 computers • Queue on each end • Can send both ways (“Full Duplex”) • Rules for communication? “protocol” – Inside a computer: • Loads/Stores: Request (Address) & Response (Data) • Need request & response signaling – Name for standard group of bits sent: packet ENGS 116 Lecture 19 6 A Simple Example • What is the format of a packet? (Protocol) – Fixed? Number of bytes? Request/ Response Address/Data 1 bit 32 bits 0: Please send data from address 1: Packet contains data corresponding to request ENGS 116 Lecture 19 Questions About Simple Example • What if more than 2 computers want to communicate? – Need computer address field (destination) in packet • What if packet is garbled in transit? – Add error detection field in packet (e.g., CRC) • What if packet is lost? – More elaborate protocols to detect loss (e.g., NAK, ARQ, time outs) • What if multiple processes per machine? – Queue per process • Questions such as these lead to more complex protocols and packet formats 7 ENGS 116 Lecture 19 8 A Simple Example Revisited • What is the format of a packet? – Fixed? Number of bytes? Request/ Response Address/Data CRC 2 bits 32 bits 4 bits 00: 01: 10: 11: Request—Please send data from address Reply—Packet contains data corresponding to request Acknowledge request Acknowledge reply ENGS 116 Lecture 19 9 Additional Background • Connection of 2 or more networks: Internetworking • 3 cultures for 3 classes of networks – SAN: server (storage) networks, performance – LAN: workstations, cost – WAN: telecommunications, long range • Cost • Performance (BW, latency) • Reliability ENGS 116 Lecture 19 10 Interconnections (Networks) • Examples: – SAN networks (infiniband): 100s nodes; ≤ 100 meters per link – Local Area Networks (Ethernet): 100s nodes; ≤ 1000 meters – Wide Area Network (ATM): 1000s nodes; ≤ 5,000,000 meters Node Node Node SW Interface SW Interface SW Interface HW Interface HW Interface HW Interface Link Link Link Interconnect Node ... SW Interface HW Interface ... Link ENGS 116 Lecture 19 11 Software to Send and Receive • SW Send steps 1: Application copies data to OS buffer 2: OS calculates checksum, starts timer 3: OS sends data to network interface HW and says start • SW Receive steps 3: OS copies data from network interface HW to OS buffer 2: OS calculates checksum, if matches send ACK; if not, deletes message (sender resends when timer expires) 1: If OK, OS copies data to user address space and signals application to continue • Sequence of steps for SW: protocol – Example similar to UDP/IP protocol in UNIX ENGS 116 Lecture 19 12 Network Performance Measures Node Node ... SW Interface SW Interface HW Interface HW Interface Overhead Overhead Link Link Bandwidth ... Link Interconnect Bisection Bandwidth Latency Link Bandwidth ENGS 116 Lecture 19 13 Universal Performance Metrics Sender Sender Overhead Transmission time (size ÷ bandwidth) (processor busy) Time of Flight Transmission time (size ÷ bandwidth) Receiver Overhead Receiver Transport Latency (processor busy) Total Latency Total Latency = Sender Overhead + Time of Flight + Message Size ÷ BW + Receiver Overhead Includes header/trailer in BW calculation? ENGS 116 Lecture 19 14 Simplified Latency Model • Total Latency ≈ Overhead + Message Size / BW • Overhead = Sender Overhead + Time of Flight + Receiver Overhead • Example: show what happens as we vary the following – Overhead: 1, 25, 500 µsec – BW: 10, 100, 1000 Mbit/sec (factors of 10) – Message Size: 16 Bytes to 4 MB (factors of 4) • If overhead is 500 µsec, how big a message is needed to get > 10 Mb/s of bandwidth? ENGS 116 Lecture 19 15 Overhead, Bandwidth, Size Effective Bandwidth (Mbits/sec) 1000 bw1000 o1, bw1000 o500, bw1000 o25, bw1000 100 bw100 o1, bw100 o500, bw100 10 bw10 o500, bw10 o1, bw10 o1, bw100 1 o1, bw1000 o25, bw10 o25, bw100 o25, bw1000 0.1 o500, bw10 o500, bw100 o500, bw1000 0.01 Message Size (bytes) ENGS 116 Lecture 19 16 Measurement: Sizes of Message for NFS 100% 90% Msgs Cumulative % 80% 70% Why? Bytes 60% 50% 40% 30% 20% 10% 0% 0 1024 2048 3072 4096 Packet size 5120 • 95% messages, 30% bytes for packets ≤ 200 bytes • > 50% data transferred in packets = 8KB 6144 7168 8192 ENGS 116 Lecture 19 17 HW Interface Issues • Where to connect network to computer? – Cache consistent to avoid flushes? ( memory bus) – Latency and bandwidth? ( memory bus) – Standard interface card? ( I/O bus) – MPP memory bus; LAN, WAN I/O bus CPU Network Network $ I/O Controller L2 $ Memory Bus Memory Bus Adaptor I/O Controller I/O bus ideal: high bandwidth, low latency, standard interface ENGS 116 Lecture 19 18 SW Interface Issues • How to connect network to software? – Programmed I/O? (low latency) – DMA? (best for large messages) – Receiver interrupted or received polls? • Things to avoid – Invoking operating system in common case – Operating at uncached memory speed (e.g., check status of network interface) ENGS 116 Lecture 19 19 CM-5 Software Interface – Time per poll 1.6 secs; time per interrupt 19 secs – Minimum time to handle message: 0.5 secs – Enable/disable 4.9/3.8 secs • As rate of messages arriving changes, use polling or interrupt? – Solution: Always enable interrupts, have interrupt routine poll until no messages pending – Low arrival rate interrupt – High arrival rate polling Overhead 100 90 80 message overhead (µsecs) • CM-5 example (MPP) 70 60 Polling 50 40 30 Interrupts 20 10 0 0 10 20 30 40 50 60 70 80 message interarrival (µsecs) Time between messages 90 100 ENGS 116 Lecture 19 20 Network Media Twisted Pair: Coaxial Cable: Fiber Optics Transmitter – L.E.D – Laser Diode light source Copper, 1mm thick, twisted to avoid antenna effect (telephone) Plastic Covering Braided outer conductor Insulator Copper core Air Total internal reflection Silica Used by cable companies: high BW, good noise immunity Receiver – Photodiode 3 parts are cable, light source, light detector. ENGS 116 Lecture 19 21 Connecting Multiple Computers • • Shared Media vs. Switched: pairs communicate at same time, “point-to-point” connections Aggregate BW in switched network is many times that of shared – point-to-point faster since no arbitration, simpler interface • Shared Media (Ethernet) Node Node Node Switched Media (CM-5, ATM) Node Node Arbitration in shared network? – Central arbiter for LAN? – Listen to check if being used (“Carrier Sensing”) – Listen to check if collision (“Collision Detection”) – Random resend to avoid repeated collisions; not fair arbitration; – OK if low utilization Switch Node Node (a.k.a. data switching interchanges, multistage interconnection networks, interface message processors) ENGS 116 Lecture 19 22 Switch Topology • Structure of the interconnect • Determines – Degree: number of links from a node – Diameter: max number of links crossed between nodes – Average distance: number of hops to random destination – Bisection: minimum number of links that separate the network into two halves (worst case) • Warning: these three-dimensional drawings must be mapped onto chips and boards which are essentially two-dimensional media – Elegant when sketched on the blackboard may look awkward when constructed from chips, cables, boards, and boxes (largely 2D) ENGS 116 Lecture 19 23 A Simple Example Figure 8.15 A ring network topology. ENGS 116 Lecture 19 24 b) Linear array a) Bus c) Star e) Tree d) Ring f) Near-neighbor mesh h) 3–cube (hypercube) g) Completely connected Examples of Static Interconnection Network Topologies ENGS 116 Lecture 19 Figure 8.16: Network topologies that have appeared in commercial MPPs. 25 ENGS 116 Lecture 19 26 Important Topologies Type Degree Diameter Avg Dist Bisection N = 1024 Diam Avg D 1D mesh ≤2 N-1 N/3 1 2D mesh ≤4 2(N1/2 - 1) 2N1/2 / 3 N1/2 63 21 3D mesh ≤6 3(N1/3 - 1) 3N1/3 / 3 N2/3 ~ 30 ~ 10 Ring 2 N/2 N/4 2 2D torus 4 N1/2 N1/2 / 2 2N1/2 32 16 Hypercube n n = LogN n/2 N/2 10 5 ENGS 116 Lecture 19 27 0 0 0 0 0 1 0 0 0 3 2 1 1 0 1 1 2 0 2 1 3 0 1 2 3 4 5 6 7 8 9 10 11 12 Figure 8.14 A fat-tree topology for 16 nodes. 0 13 3 14 1 15 ENGS 116 Lecture 19 Figure 8.13 Popular switch topologies for eight nodes. 28 ENGS 116 Lecture 19 29 b) 8 8 Baseline a) Crossbar switch = processor = switch Examples of dynamic interconnection network topologies