Download Document

TDC 564
Local Area Networks
Lecture 8
IP-based Storage Area
Network
11/05/07
TDC564-08
1
Course Outline


IP over FC (RFC 2625)
IP-SAN


iSCCI (RFC 3720)
IP and FC-SAN Interworking
 FC
Encapsulation (RFC 3643)
 FCIP (RFC 3821) – FC over IP
 iFCP (RFC 4172)
11/05/07
TDC564-08
2
RFC 2625 – IP and ARP over FC


FC supports multiple higher layer protocols, and
SCSI is the most widely used one.
Needs/Applications of IP over FC:
Access data in SAN from IP-based servers
 interworking between NAS and SAN


RFC 2625 addresses two issues.
A scheme to encapsulate IP and ARP packets inside
the FC frame (as the FC payload)
 A procedure to resolve the address mapping

11/05/07
TDC564-08
3
IP over FC (RFC 2625)
App-1: accessing SAN from IP-based servers
SAN
FC-based
Storage Device
FC/IP Gateway
Data
Data
IP
IP
IP
IP
L2
L2
RFC 2625
RFC 2625
PHY
PHY
FC
FC
FC
SAN
11/05/07
TDC564-08
4
IP over FC (RFC 2625)
(App-2: interworking between SAN and NAS)
SAN
NAS-based
Storage Device
FC/IP Gateway
FC/IP Gateway
Data
Data
IP
IP
IP
IP
IP
IP
L2
L2
RFC 2625
RFC 2625
L2
L2
PHY
PHY
FC
FC
PHY
PHY
FC
SAN
11/05/07
TDC564-08
5
FC Address Resolution

Three steps schemes




ARP - IP to MAC
MAC => World-Wide Port Name (WW_PN)
FC-ARP - WW_PN => Port_ID
Optional Header is used for Network Header by RFC 2625
D_NAA (4) Dst. Network Address (60)
S_NAA (4)
16 bytes
Src. Network Address (60)
Format of Network Header
NAA: Network Address Authority
unused (12 bits) IEEE 802.3 MAC Address (48 bits)
Format of Network Address (WW_PN)
11/05/07
TDC564-08
6
RFC 2625: IP and ARP over FC (1)
ARP Request (broadcast) A
- Sender HW (MAC) Address = HW(A)
- Sender IP Address = IP(A)
- Target HW (MAC) Address = ??
- Target IP Address IP(B)
FC SAN
FC switch
C
B
11/05/07
TDC564-08
D
7
RFC 2625: IP and ARP over FC (2)
ARP Reply (unicast):
- Sender HW (MAC) Address = HW(B)
- Sender IP Address = IP(B)
- Target HW (MAC) Address = HW(A)
- Target IP Address IP(A)
A
C
B
11/05/07
TDC564-08
D
8
RFC 2625: IP and ARP over FC (3)
FC-ARP Request:
- Requester N_Port_Name (WW_PN)
- Requester N_Port_ID
- Responder N_Port_Name (WW_PN)
- Responder N_Port_ID = ??
A
C
B
unused (12 bits) IEEE 802.3 MAC Address (48 bits)
11/05/07
TDC564-08
D
WW_PN
9
RFC 2625: IP and ARP over FC
(4)
FC-ARP Reply:
A
- Requester N_Port_Name
- Requester N_Port_ID
- Responder N_Port_Name
- Responder N_Port_ID
C
B
11/05/07
TDC564-08
D
10
Advantages of IP for SAN
11/05/07
TDC564-08
11
IP Network Capabilities
11/05/07
TDC564-08
12
IP-SAN Protocols
11/05/07
TDC564-08
13
IP - SAN
IP
IP
FC -SAN
Storage Devices
11/05/07
TDC564-08
14
IP - Storage Area Network (SAN)




IP storage networking – carrying storage traffic over IP
Uses TCP, a reliable transport for delivery
Can be used for local data center and long haul applications
Two primary IETF protocols/standards:
iSCSI – Internet SCSI – allows block storage to be accessed
over a TCP/IP network as though it were locally attached

IP
TCP
iSCSI
SCSI
Data
FCIP – Fibre-Channel-over-IP – used to tunnel Fibre Channel frames
over TCP/IP connections

IP
11/05/07
TCP
TDC564-08
FCIP
FC SCSI
Data
15
Internet SCSI (iSCSI)





iSCSI is a proposed industry standard that allows SCSI block
I/O protocols (commands, sequences, and attributes) to be sent
over a network using the popular TCP/IP protocol.
A way to access storage across an IP network as though it was
locally attached.
Transports SCSI protocol commands and data across an IP
network
Cisco and IBM co-authored original iSCSI protocol draft
iSCSI Protocol is a standard maintained by the IETF
 IP Storage (IPS) Working Group
 RFC 3720
11/05/07
TDC564-08
16
iSCSI
11/05/07
TDC564-08
17
iSCSI over TCP/IP
SCSI
Command
Command
TCP Header
11/05/07
Data
Command
Data
TCP Payload
TDC564-08
18
iSCSI Applications



Connectivity : iSCSI can be used for DAS or
SAN connections.
iSCSI capable devices could be placed on an
existing LAN (shared with other applications) in
a similar way to NAS devices.
iSCSI capable devices could be attached to a
LAN which is dedicated to storage I/O (in other
words an IP SAN), or even on a LAN
connected to only one processor (like a DAS).
11/05/07
TDC564-08
19
iSCSI Protocol Stack
Software iSCSI
Data
SCSI
iSCSI
TCP
IP
iSCSI TCP Offload
iSCSI
Engine
Host Bus Adapter (HBA)
(TOE)
Data
Data
SCSI
SCSI
iSCSI
Host
Processing
Driver
Driver
Ethernet
Driver
TCP
IP
Ethernet
iSCSI
TCP
IP
Ethernet
PHY
PHY
PHY
Adapter
Card
Host Bus Adapter (HBA)
11/05/07
TDC564-08
20
iSCSI Performance Consideration
11/05/07
TDC564-08
21
iSCSI Benefit
11/05/07
TDC564-08
22
Host Bus Adapter (Fibre Channel)
QLogic PCI-EXPRESS 4Gb 1PT
Fibre Host Bus Adapter – STD FW
• PCI-Express
• • 1 Port (Fibre Channel)
• Data Rate: 4Gb
• Standard Firmware
Price Quote: $951
Source: http://www.cdw.com
11/05/07
TDC564-08
23
Host Bus Adapter (iSCSI)
10/100/1000BaseT
PCI
11/05/07
The QLE4060C iSCSI Host Bus Adapter
(HBA) provides connectivity to SANs
over Ethernet and TCP/IP network
infrastructures for PCI Express bus-based
servers. Using a widely deployed and
familiar networking technology, iSCSI can
lower the total cost of ownership to
better manage and support storage
networking.
Source: http://www.cdw.org
Price quote: $815
TDC564-08
24
iSCSI Storage
Promise Technology VTrak M-Class iSCSI RAID
Price quote: $4,929
Source: //www.cdw.com
Network: Gigabit Ethernet (2)
Protocol: TCP/IP, iSCSI
Capacity: 16-drive bay (500G – 1TB per drive)
11/05/07
TDC564-08
25
Data
SCSI
iSCSI
TCP
IP
Layer2
PHY
iSCSI Application
SCSI
IP
iSCSI
TCP
IP
Layer2
PHY
FCP
FC 0-2
Data
SCSI
FCP
FC 0-2
Storage
Device
iSCSI
Gateway
Server
Cisco iSCSI
Configuration Example
11/05/07
TDC564-08
26
Multi-TCP Sessions
11/05/07
TDC564-08
27
HBA: Multi-Path I/O
LUN: Logical [Storage] Units
11/05/07
TDC564-08
28
HBA iSCSI vs. Software iSCSI

No performance advantage of using iSCSI HBA
Throughput
 Round trip time


Significantly higher cost of iSCSI HBA


What are the advantages of iSCSI HBA?


$800 vs. $100 (Server NIC)
Answer: CPU Utilization (off-load)
Reference:
http://download.intel.com/network/connectivity/products/iSCSIcomp0407.pdf
11/05/07
TDC564-08
29
Is IP-SAN similar to
NAS?
What are the advantages, if
any, of IP-SAN vs. NAS?
11/05/07
TDC564-08
30
Performance Analysis: iSCSI vs. NAS
(software based – no HBA)
iSCSI
NFS
http://technomagesinc.com/pdf/ip_paper.pdf
11/05/07
TDC564-08
31
Sequential Read/Write Tests
Conclusion:
1. Comparable performance in character read/write and block write
2. Significant advantage of iSCSI in block read (20-25%)
11/05/07
TDC564-08
32
Small Files Read/Write
11/05/07
TDC564-08
33
IOGen Test (Emulation of Database)
11/05/07
TDC564-08
34
FC-SAN vs. iSCSI

Since the iSCSI appliance attaches to the existing Ethernet
network, NAS and iSCSI are very similar in network architecture




However, the performance would be significantly different.
Both iSCSI and SAN use Block I/O to transport data, whereas
NAS uses File I/O.
SAN offers better performance (c.f. NAS), but is more expensive
and requires a higher skill set to implement. iSCSI and NAS
offer better pricing and skills may already be in place to
implement them.
Both SAN and iSCSI offer the performance benefit of Block
I/O.
11/05/07
TDC564-08
35
Case Study: FC-SAN vs. iSCSI
http://www.promise.com/marketing/whitepaper/file/2_MClassiSCSI%20SCSI%20and%20Fibre-10-07-05.pdf
11/05/07
TDC564-08
36
 4G bps
11/05/07
TDC564-08
37
FC over IP
• The encapsulation of Fibre Channel frames into IP packets and
tunneled through an existing TCP/IP network infrastructure, in order
to connect remote islands of FC-based SANs
FC SAN
IP
IP
TCP
FCIP
FC SAN
FC SCSI
Data
11/05/07
TDC564-08
38
FC Frame Encapsulation (RFC 3643)

Describes the common Fibre Channel (FC)
frame encapsulation format and a procedure for
the measurement and calculation of frame
transit time through the IP network
Encapsulation Header SOF FC Frame Body EOF
11/05/07
TDC564-08
39
FC Encapsulation Header
Q: Why is time stamp needed in the FC encapsulation header?
11/05/07
TDC564-08
40
FC
Encapsulation
11/05/07
TDC564-08
41
FCIP (RFC 3821)
Fibre Channel Over TCP/IP (FCIP) describes mechanisms that
allow the interconnection of islands of Fibre Channel storage area
networks over IP-based networks to form a unified storage area
network in a single Fibre Channel fabric
11/05/07
TDC564-08
42
FCIP Protocol Stacks
11/05/07
TDC564-08
43
FC over IP
SAN
Data
IP
SAN
an IP tunnel for FC-based SAN
Application: interconnect SAN over IP-WAN.
Data
SCSI
SCSI
FCP
FCP
FCIP
FCIP
FCP
FCP
FC 0-2
FC 0-2
TCP
TCP
FC 0-2
FC 0-2
11/05/07
IP
IP
IP
L2
L2
L2
PHY
PHY
PHY
TDC564-08
44
FCIP Configuration Example
11/05/07
TDC564-08
45
Internet Fiber Channel Protocol (iFCP)



11/05/07
iFCP – RFC 4172 (Sept. 2005)
iFCP is a protocol that uses the FCP over
IP networks to interconnect FC-based
storage devices.
It is considered a gateway protocol, while
FCIP is considered a tunnel protocol.
TDC564-08
46
iFCP (RFC 4172)



iFCP specifies an architecture and a gateway-to-gateway
protocol for the implementation of fibre channel fabric
functionality over an IP network.
This functionality is provided through TCP protocols
for fibre channel frame transport and the distributed
fabric services specified by the fibre channel standards.
The architecture enables internetworking of fibre
channel devices through gateway-accessed regions with
the fault isolation properties of autonomous systems
and the scalability of the IP network
11/05/07
TDC564-08
47
iFCP Network Example
11/05/07
TDC564-08
48
iFCP (RFC 4172)
F-port
F-port
IP
iFCP GW
Data
iFCP GW
Gateway approach
Application: interconnect SAN over IP-WAN.
SCSI
FCP
FC 0-2
11/05/07
Data
SCSI
FCP
FC 0-2
FCP
iFCP
iFCP
TCP
TCP
IP
IP
IP
L2
L2
L2
PHY
PHY
PHY
TDC564-08
FC 0-2
FCP
FC 0-2
49
iFCP
11/05/07
TDC564-08
50
FCIP vs. iFCP
http://whatis.techtarget.com/magPrintFriendly/0,293813,sid35_gci1096482,00.html
11/05/07
TDC564-08
51
FCIP vs. iFCP
McDATA
11/05/07
TDC564-08
52
SAN Performance Comparison
http://www.netapp.com/library/tr/3423.pdf
11/05/07
TDC564-08
53
Summary

Needs for large storage – continual growth


From dedicated solution to network-based solution



SCSI over FC
SCSI over IP (iSCSI)
FC and IP interworking protocols


11/05/07
It is an IP world!
SCSI is the protocol for block data transmission


DAS => NAS => SAN => IP-SAN
Convergence of SAN and IP-LAN/WAN


109 (G) => 1012 (T) => 1015 (P) => 1018 (E) …….
IP over FC
FC over IP (FCIP) and iFCP
TDC564-08
54