Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
IP over InfiniBand: Connected Mode Vivek Kashyap [email protected] © 2002 IBM Corporation IETF-55: IPoIB Connected Mode IP over InfiniBand Modes Unreliable Datagram AD review comments IBTA 1.1 updates Reliable Connected Unreliable Connected Reliable Datagram 2 Vivek Kashyap, IBM IETF-55: IPoIB Connected Mode IPoIB non-UD options RC RD Large MTU Chart Title 2^31 bytes 2-4K links 2^31 bytes 2-4K links 2^31 bytes 2-4K links Automatic Path Migration Yes Yes Yes Reliability Yes No Message lost Yes RDMA Yes RDMA Write Yes No. of messages outstanding 3 UC One Vivek Kashyap, IBM IPoIB over Connected Mode Reliable Connected Unreliable Connected © 2002 IBM Corporation IETF-55: IPoIB Connected Mode IPoIB Subnet IPoIB-Connected subnet is formed over a Non-Broadcast Multiple Access network IPoIB-UD is over an abstract broadcast link Node A A IPa IPp IPs Node D IPb IPd IPq Node B IPr IPc Node C IB switched network IPoIB UD broadcast domain IPoIB Connected network 5 Vivek Kashyap, IBM IETF-55: IPoIB Connected Mode Connection Setup: Address Resolution The Connection can be setup only when the peer’s GID is known – Manually configured (or Permanent Virtual Circuit) • Inverse ARP – Dynamically resolved • ARP Server as in Classical IP over ATM (RFC2225) or • IPoIB address resolution – Use a UD QP • may reuse IPoIB-UD QP – Define ARP server link address same as IPoIB-UD Broadcast GID • FF12:<401B or 601B>:<P_Key>::< 255.255.255.255 or 1> 6 Vivek Kashyap, IBM IETF-55: IPoIB Connected Mode Connection Setup: Option 1 SendFrom IPb to IPa IPb creates ARP/ND packet with local hardware address of : [RC|UC]:FFFFFF:GIDb A bit indicates RC or UC to let the peer know of the transmitter’s view of the subnet. IPa receives the packet Determine PathRecord to GIDb Send reply over UD-QP with GIDa IPb now creates the IB connection to IPa Service ID of: 1:Protocol or FF:0:0:0:0:0:0 – Protocol indicates IPv4 or IPv6 – FF indicates both IPv4 and IPv6 packets will use this connection – Private Data includes the IP address that was resolved 7 Vivek Kashyap, IBM IETF-55: IPoIB Connected Mode Address Resolution: Option 2 SendFrom IPb to IPa IPb creates ARP/ND packet with local hardware address of : [RC|UC]:FFFFFF:GIDb A bit indicates RC or UC to let the peer know of the transmitter’s view of the subnet. IPa receives the packet Determine PathRecord to GIDb (same as for IPoIB-UD) Creates QPc of type RC/UC as indicated – may reuse existing QP Send reply with QPc and GIDa IPb now creates the IB connection to the peer Service ID of: 1:QPc:0:0:0:0:0:0 IPa updates ARP/ND entry with QP value received in IB connection 8 Vivek Kashyap, IBM IETF-55: IPoIB Connected Mode Encapsulation Connection can carry IPv4 or IPv6 or both packets. Therefore – Type of packet is known – Type of packet is distinguishable from the IP header Two options: – IP packets directly encapsulated in the IB connection mode payload – IP packets encapsulated with the ‘Ethertype’ indication as in IPoIB-UD Address resolution over UD QP needs to follow IPoIB-UD encapsulation 9 Vivek Kashyap, IBM IETF-55: IPoIB Connected Mode MTU MTU Negotiated per IB connection Value requested sent in private data of CM REQ message to peer B If not acceptable counter value in CM REP message to peer A If not acceptable REJ with desired value in private data to peer B 10 Vivek Kashyap, IBM IPoIB Connected Mode Vivek Kashyap [email protected] © 2002 IBM Corporation