Download No Slide Title - BNRG - University of California, Berkeley

Storage Wide-Area Networks
(SWANs)
Randy H. Katz
Computer Science Division
Electrical Engineering and Computer Science Department
University of California, Berkeley
Berkeley, CA 94720-1776
Storage Networks
• Primary goal is to share storage among
computers in complex, heterogeneous
environments, with PCs, workstations, file
servers, and mainframes
• Storage can be:
– Direct attached (e.g., host bus adapter/HBA)
– Network attached (via a file server)
– Channel attached (primarily fibre channel, but
also IBM SSA)
Storage E.g, Hitachi, EMC, IBM Storage Arrays
Networks
Fibre Channel Loops
Dual Ported Disks
And Controllers
Remote
Storage
Manager
DI
DI
DI
DI
Cache
Workstation
Crossbar
Interconnect
Mainframe
Fibre Channel
Or ESCON
Cache
Cache
Cache
HI
HI
HI
HI
Multiple Host
Interconnections
Storage Networks
• Major development in 1990s: storage networks
– Native FC is a point-to-point or loop/string-oriented
(“arbitrated”) method of interconnection; 1 Gbps, extend
up to 10 km
– Emergence of FC “fabrics”: FC switches arranged
hierarchically to enable connectivity between any host and
any storage device (e.g., Brocade Networks dominates this
product space)
– Standardize FC protocol stack lives on top of such fabrics:
device naming, transport, CoS, etc.
– SCSI-3 protocol over FC fabrics
– Gigabit ethernet now emerging in SAN environment
Basic Attached Storage
Host
Disk Interface (DI)
OS
Allocation
Table
Disk,
Cylinder,Track,
Sector
• Device attached by SCSI HBA or channel interface
• Host manages the file-to-block mapping
Network-Attached Storage (NAS)
aka Network File Service
Host
Host
Host
Network
Interface
(NI)
File Name,
Offset, Length
Network
File
Server
LAN
Network
Interface
(NI)
OS
Network
Interface
(NI)
• Mapping from File to Block done in
network-attached File Server, not host
Network-Attached Secure
Devices (NASD)
Gibson@CMU: Research project on
device embedded protocol stack,
authentication
Host
OS
Host
Host
Network
Interface
(NI)
Disk,
Cylinder,
Track,
Sector
File Name,
Offset, Length
LAN
Network
Interface
(NI)
Network
File
Server
OS
Network
Interface (NI)
Network-attached
Secure Device (NASD)
Storage Platforms
“Storage Virtualization”
LUN = Logical Unit
Main
Frame
OS
Main
Frame
Work
Station
Channel
Interface
LUN,
Offset,
Length
Disk
Storage
Subsystem
LUN
To
PHY
Logical disk mapping onto
underlying physical disks on
logical block to physical
block basis
NAS
NAS distinguished by an exported
Network File System interface over a standard
Local Area Network-based transport
Host
Host
Host
File
Server
Network
Interface
(NI)
LAN
Network
Interface
(NI)
File Name,
Offset, Length
File
Server
Network
Interface
(NI)
File
Server
NAS + SAN
SAN distinguished by a block-oriented interface;
Usually implemented across a channel-oriented fabric
Host
Host
Host
File
Server
Network
Interface
(NI)
LAN
Network
Interface
(NI)
Disk
CI
File Name,
Offset, Length
File
Server
Network
Interface
(NI)
File
Server
PHY Device,
Cyl, Trk, Sector
Main
Frame
Channel
Storage
Interface Subsystem
LUN,
Offset, Length
SAN
CI
CI
Main
Frame
LUN,
Offset,
Length
CI
Optical
Disk
Storage
Subsystem
Tape
Storage
Subsystem
NAS + SAN + SWAN
Now extend the NAS or the
SAN over a wide-area
network transport …
WAN
NOTE: wide-area
SAN is new idea
Gate
way
Host
Host
Host
Network
Interface
(NI)
LAN
Network
Interface
(NI)
Gate
way
Network
Interface
(NI)
File
Server
DSS
PHY Device,
Cyl, Trk, Sector
Main
Frame
File
Server
File
Server
SAN
FS
Remote SAN
Disk
CI
File Name,
Offset, Length
LAN
Channel
Storage
Interface Subsystem
LUN,
Offset, Length
SAN
CI
CI
Main
Frame
LUN,
Offset,
Length
CI
Optical
Disk
Storage
Subsystem
Tape
Storage
Subsystem
Storage Domain
File/Record Subsystem
Host
Block
Aggregation
SAN
Device
Block Subsystem
Discovery, Monitoring
Resource Mgmt, Configuration
Security, Billing
Redundancy Mgmt, Back-up
High Availability, Fail-over
Capacity Planning
Service Subsystem
Shared Storage
Reference Model
Application
SAN Reference Model
Block-oriented
SAN
File
Application
FS
Block
Host-based
SAN
SAN-based
DA
Device-based
SAN Reference Model
Application
File
Host
NAS
Storage
Host
LAN
FS
Block
Host
NAS
SAN
Device
SAN Reference Model
File
Application
Host
Host
s/w Host
raid
Host
LAN
Block
NAS
Head
SAN
DA
Heterogeneous
Storage
Environment
FS
Host
NAS
SAN
Device
Seven Layer Stack
IP
7
Application
6
Presentation
5
Session
4
Transport
3
Network
IP
2
Data Link
LAN, MAN, WAN
1
Physical
Sw GigaE
NFS, CIFS
FTP, SNMP,
TFTP, Telnet,
FCP, SCSI-3
NFS, CIFS
FTP, SNMP,
TFTP, Telnet,
FCP, SCSI-3
TCP, UDP
TCP, UDP
Phy
FC
SCSI-3
VI
IP
FC-4
Protocol I/F
IP
FC-3 Encrypt/
Authentication
Mac Client/
FC-2 Framing, FC,
Control
Class of Service
Phy
FC-1 Encoding,
Link Control
FC-0 Phy
Fibre Channel Protocol Stack
• FC-0: gigabit physical layer
• FC-1: data encoding and link layer control
• FC-2: segmentation/reassembly of data frames,
flow control, class of service
• FC-3: common services, e.g., encryption
• FC-4: “upper layer protocol” upon which SCSI-3 or
IP can run
• Contrast with Gigabit Ethernet
– Ethernet framing, VLAN tagging, frame prioritization (8
levels), link aggregation, 1.25 gbps
– IP + Gigabit Ethernet emerging for SANs
SCSI-3 Terminology
Target
Initiator
Request
Application
Client
Response
Delivery Subsystem
(e.g., Fibre Channel or
Serial SCSI over IP)
LUNs
Device
Server
SCSI
Client-Server
Model
FC over IP (FCIP)
Tape
Storage
Subsystem
Server
FC
Over
IP
FC
Switch
Server
JBOD
Tape
Storage
Subsystem
Server
IP Network
WAN
Tunnel Session
FC
Over
IP
FC
Switch
Server
JBOD
• IETF IP Storage (IPS) working group
– Recall FC fabrics developed in context of machine
room/building-scale interconnect (e.g., no congestion control!)
• FC time outs in wide-area? Flow control interaction? QoS?
• How does bridging actually work for FC e2e management?
Internet FC Protocol (iFCP)
FC Device
Address
IP
Address
Mapping
FC_Device
N_Port
F_Port
iFCP layer
FCP Portal
iFCP gateway
region
•
•
•
•
FC Traffic
Control Data
iFCP Frames
FC_Device
N_Port
F_Port
iFCP layer
FCP Portal
IP Network
iFCP gateway
region
Gateway to gateway protocol, sessions rather than tunnels
TCP for congestion control, error detection, recovery
Plug FC devices directly into iFCP switches
Session and naming semantics
iFCP Services
• 24-bit N-Port Address: <Domain, Area, Port>
• iSNS: Internet Storage Name Service—discovery
and management protocol for IP storage networks
(IPNSP)
• Protocol specification includes address translation
feature to allow remote storage devices to be
assigned a local, FC fabric compliant address
– Local commands executed locally on the fabric
– Remote commands executed on top of TCP connections
• Error Detection/Time Outs
• Security
Other Protocols
• Metro Fibre Channel Protocol (mFCP)
– FCP over IP using UDP rather than TCP (link
layer flow control and pacing)
• Internet SCSI (iSCSI)
– IP to the storage device
– Serial SCSI block data transfer over IP (SCSI
Access Method Command Set—SAM)
– IPSec, command/data ordering, steering to
application memory
iSNS
• Discovery Process
– Device registration
– WWN or iSCSI names
– Zoning/discovery domains
• iSNS objects
– Portals
– Storage Port
– Storage Nodes
Storage Applications
• Data Centers
– Server clustering
– Storage centralization, consolidation, management
– LAN-free back-up
• IP Storage for Remote Applications
–
–
–
–
Remote back-up
Remote mirroring
Disaster recovery
Content distribution
Rhapsody Networks, Inc.
• Storage Application Director
– “Alteon box for storage networks”
– Peek into storage packets traversing fabric
(“deep frame classification”) and invoke code—
for encryption/decryption, mirroring, LUN
mappings, etc.
– Per port software processing and cut-through
fabric routing
– Data copy engine, table lookup engine, in-transit
I/O mods, data escrow/trap to software for
complex errors or event processing
– Intelligent queue management