Download Storage Fundamentals 101

SAN Fundamentals
2014 LENOVO . ALL RIGHTS RESERVED.
Agenda












2
Distributed Model of data flow
Direct Attach Storage
SCSI Protocol
SCSI addressing / subaddressing
Fiber Channel
Centralizad Model of data flow
Storage Area Networks
Topologies
Management of SAN
Zoning
ISL
Architecture
2014 LENOVO. ALL RIGHTS RESERVED.
Data And Disks

Data is written to, and retrieved from, a disk drive
 Typically via local application
or server application

Applications / file systems utilize block-level I/O
 Send/retrieve data via I/O blocks
0110100101011101010001

3
Users utilize file-level I/O
 Send / retrieve data files
2014 LENOVO. ALL RIGHTS RESERVED.
Yesterday’s View of Information
4
2014 LENOVO. ALL RIGHTS RESERVED.
Direct Attached Storage
DAS Device
5
2014 LENOVO. ALL RIGHTS RESERVED.
DAS Device
Direct Attached Storage



Storage is directly connected to a single server
 SCSI, SAS, iSCSI, Fibre Channel
 Block-level I/O
Internal drives
 With or without RAID protection
External drives
 Storage system
 Controller-based RAID engine
6
2014 LENOVO. ALL RIGHTS RESERVED.
External DAS
Application
Server
Motherboard
HA
Controllers
External storage system connected
via host adapter
• Controller-based RAID
7
2014 LENOVO. ALL RIGHTS RESERVED.
Direct Attached Storage


Connects to Server using SAS or SCSI
Maybe JBOD or RAID
Server
DAS Device
8
2014 LENOVO. ALL RIGHTS RESERVED.
Direct Attached Storage
 Issues
 Limited distance
Server
9
2014 LENOVO. ALL RIGHTS RESERVED.
Direct Attached Storage

Issues
 Limited distance
 Limited number of devices (SCSI)
 Limited redundancy
Server
10
2014 LENOVO. ALL RIGHTS RESERVED.
Direct Attached Storage
The point−to−point topology in DAS
11
2014 LENOVO. ALL RIGHTS RESERVED.
Direct Attached Storage
Internal DAS Advantages



Low-cost
Drives are typically included with the server
 Immediate utilization
Some data protections
External DAS Advantages




12
Better than data stored on clients
Increased availability and performance
 RAID protections
Increased features and functionality
Scalability
2014 LENOVO. ALL RIGHTS RESERVED.
Direct Attached Storage
DAS Disadvantages





13
Each server has its own storage device ($$)
Management is difficult
 Reduced Productivity
Expensive to backup
Wasted storage space
Difficult to share data
2014 LENOVO. ALL RIGHTS RESERVED.
Direct Attached Storage
Shared DAS


14
Storage is directly connected to a single server
 SAS, iSCSI, Fibre Channel
Storage system consist of RAID controllers
and drive enclosures
 RAID engine is controller-based
2014 LENOVO. ALL RIGHTS RESERVED.
Host Bus Adapter
15
2014 LENOVO. ALL RIGHTS RESERVED.
Host Bus Adapter
The initiating device is a device that looks for and communicates with
target devices.
It is commonly referred to as the host bus adapter (HBA), and it resides
in the server or client workstation.
The significance of the HBA is that it actually is an active device that
seeks out its targeted pair to communicate with so as to achieve a
file transfer.
The HBA has the ability to monitor its path to its targeted pair.
If, for some reason, it loses contact with the target, the initiator will
switch to an alternate target.
In order for this event to occur, one level of redundancy is put in place
.
whereby there are dual initiators, dual switches, and mirrored disks
16
2014 LENOVO. ALL RIGHTS RESERVED.
What Does SCSI Stand For?
 Small Computer System Interface (SCSI)
SCSI is a protocol for connecting computers with external
devices for data management or data protection
It defines:
 Commands: These are standards that define specific
command sets for either all SCSI devices, or for particular
types of SCSI devices.
 Protocols: These standards formalize the rules by which
various devices communicate and share information,
allowing different devices to work together. These standards
are sometimes said to describe the transport layer of the
interface.
 Interconnects: These are standards that define specific
interface details, such as electrical signaling methods and
transfer modes. They are sometimes called physical layer
standards as well.
17
2014 LENOVO. ALL RIGHTS RESERVED.
The SCSI interface….started it all!


An industry standard I/O Bus
Standard connectors are the same on each device
 All devices share a common bus
8-bit data bus
18
2014 LENOVO. ALL RIGHTS RESERVED.
16-bit data bus
What Types of Devices Use SCSI?







19
Disk Drives
Tape Drives
Removable Media Drives (Zip)
CD-ROM Drives
CD-R/CD-RW Drives
Optical Memory Drives
Media Changers
2014 LENOVO. ALL RIGHTS RESERVED.
Where do SCSI and SAS fit in?
Server
SAS HBA or SCSI
adapter
target
RAID sub-system
SCSI/SAS
Chip
SCSI or SAS
Controller
initiator
SCSI/ SAS
Chip
Controller
Ethernet to Client
workstations
20
2014 LENOVO. ALL RIGHTS RESERVED.
Dual Controller RAID with only one controller
in use (B not used in this example). This RAID system has
four SCSI buses with five drives on each bus.
Logical Unit Number (LUN)


21
LUNs are a type of sub-addressing supported by SAS
and SCSI
LUNs are selected through the Identify message
Controller
Controller
Host Adapter
ID 2
ID 1
ID 7
Disk Drive
Disk Drive
Disk Drive
Disk Drive
LUN 0
LUN 1
LUN 0
LUN 1
2014 LENOVO. ALL RIGHTS RESERVED.
Redundant Array of Independent Disks
22

RAID 0: Striped set without parity (Non-Redundant Array). Fastest and most
efficient level but offers no fault tolerance

RAID 1: Mirrored set without parity neither striping.
Provides fault tolerance from disk errors and failure of all but one of the drives.
Increased read performance occurs when using a multi-threaded operating
system that supports split seeks, very small performance reduction when
writing

RAID 3:Striped set with dedicated parity/Bit interleaved parity. This
mechanism provides an improved performance and fault tolerance similar to
RAID 5, but with a dedicated parity disk rather than rotated parity stripes .

RAID 5:Striped set with distributed parity. Distributed parity requires all
drives but one to be present to operate; drive failure requires replacement, but
the array is not destroyed by a single drive failure. Upon drive failure, any
subsequent reads can be calculated from the distributed parity such that the
drive failure is masked from the end user

RAID 1+0: High performance but requires double the number of drives for
mirroring of data.It allocate blocks in stripes along the disks.
2014 LENOVO. ALL RIGHTS RESERVED.
SCSI and SAS vs. Fibre Channel
SCSI
SAS
3 meters
8 meters
Fibre
Channel
Up to 10Km
Duplex
Half
Full
Full
Devices
12
Cable
Distances
Throughput
23
320 MB/s
2014 LENOVO. ALL RIGHTS RESERVED.
~ 16k with
224 -confirm
expanders
* Up to total 12Gb/s throughput with wide ports
3Gb/s*
4Gb/s
What is Fibre Channel?

24
A transport mechanism for multiple protocols
 SCSI-3 and SAS
 Internet Protocol (IP)
 others
2014 LENOVO. ALL RIGHTS RESERVED.
Why Fibre Channel?



25
Serial Transmission
 Smaller Connectors
 Eliminate Skew problems of parallel transmission
High bandwidth
 1, 2 , 4, 8 and 16 gigabits per second (Gb/s)
Big "B" versus little "b"
 Megabit is abbreviated with a lower case b (Mb)
 Megabyte is abbreviated with a capital B (MB)
2014 LENOVO. ALL RIGHTS RESERVED.
Why Fibre Channel?
 Scalable
 Large number of devices
 Greater distance
 Transport mechanism for multiple
protocols
26
2014 LENOVO. ALL RIGHTS RESERVED.
Why Fibre Channel?



27
Permits Switching
 Avoids problems of shared media and shared
bandwidth
Allows mixed speeds
 Auto-negotiating , can adjust throughput to lower
speeds
Transport mechanism for multiple protocols
2014 LENOVO. ALL RIGHTS RESERVED.
Connectivity
28
Topology
Number of Devices
Point to Point
2
Arbitrated Loop
Up to 127
Switched Fabric
Up to 16 million
2014 LENOVO. ALL RIGHTS RESERVED.
Point to Point
Storage subsystem
Computer
Memory
N_Port0
Bridge
Processor
100 MB/s
N_Port1
TX
RX
RX
TX
100 MB/s
I/O Bus
Full Duplex (Fibre Channel example): 4Gbps + 4Gbps = 8Gbps (theoretical)
29
2014 LENOVO. ALL RIGHTS RESERVED.
Multimode Fibre
150 meters
Input to fiber
Output from fiber
Multiple path-lengths, or modes, permitted by the fiber smear the shape of the pulse
30
2014 LENOVO. ALL RIGHTS RESERVED.
Single Mode Fibre
Input to fiber
Output from fiber
Single path-length, or mode, imposed by the fiber preserves the shape of the pulse
31
2014 LENOVO. ALL RIGHTS RESERVED.
Basic Configuration
Server
RAID sub-system
HBA
Fibre Channel
SCSI
Chip
Controller
SCSI
Chip
Controller
Ethernet to Client
workstations
32
2014 LENOVO. ALL RIGHTS RESERVED.
Dual Controller RAID with only one controller
in use (B not used in this example). This RAID system has
four SCSI buses with five drives on each bus.
Dual Controller Configuration
Sample SCSI/SAS subsystem
Server
RAID sub-system
SCSI
Chip
HBA
Fibre Channel
Controller
HBA
Fibre Channel
SCSI
Chip
Controller
Ethernet to Client
workstations
33
2014 LENOVO. ALL RIGHTS RESERVED.
Dual independent controllers with automatic fail-over for
continuous availability in case one controller or one fiber link
fails.
Dual Controller Configuration
Fibre and Storage subsystem
FC/SAS
Server
HBA
FC/SAS
HBA
FC/SAS
Controller
Controller
Ethernet to Client
workstations
34
2014 LENOVO. ALL RIGHTS RESERVED.
FC/SAS
RAID
Dual independent controllers with automatic fail-over for
continuous availability in case one controller or one fiber link
fails.
Connectivity at SAN
Storage Device
Fibre Channel
Server
HBA
HBA
Fibre Channel
Fibre Channel
Controller
Controller
Ethernet to Client
workstations
35
2014 LENOVO. ALL RIGHTS RESERVED.
Fibre Channel
RAID
Dual independent controllers with automatic fail-over for
continuous availability in case one controller or one fiber link
fails.
Connectivity at SAN
Servers
RAID
AB
Switch
AB
AB
Storage Area Network
Switch
AB
(SAN)
Switch
AB
AB
Switch
Storage subsystem
AB
Network Attached Storage
(NAS)
36
2014 LENOVO. ALL RIGHTS RESERVED.
N
A
S
N
A
S
N
A
S
Servers
Centralizing Information
Purchasing
Finance
Mainframe
UNIX
Information
NT
Operations
37
2014 LENOVO. ALL RIGHTS RESERVED.
NAS
Support
Centralizing Information = Value
Unused storage capacity may be easily allocated to servers as need.
Distributed Storage
38
2014 LENOVO. ALL RIGHTS RESERVED.
Centralized Storage
What is a Storage Area Network (SAN)?
39
2014 LENOVO. ALL RIGHTS RESERVED.
SAN: Storage Area Network






40
A SAN is a specialized, high−speed network attaching servers and
storage devices. It is sometimes called "the network behind the
servers.“
A SAN allows "any−to−any" connection across the network, using
interconnect elements such as routers, gateways, hubs, switches, and
directors.
It eliminates the traditional dedicated connection between a server and
storage, and the concept that the server effectively "owns and
manages" the storage devices.
It also eliminates any restriction to the amount of data that a server can
access, currently limited by the number of storage devices, which can
be attached to the individual server.
Instead, a SAN introduces the flexibility of networking to enable one
server or many heterogeneous servers to share a common storage
utility, which may comprise many storage devices, including disk, tape,
and optical storage.
The storage utility may be located far from the servers that use it.
2014 LENOVO. ALL RIGHTS RESERVED.
Block Level Access – SAN and DAS


When an application on the network requests information, the
request is handled by a server and the correct blocks of data are
returned to the client.
Databases are probably the largest example of block level data
access in the data center today.
0110100101011101010001
41
2014 LENOVO. ALL RIGHTS RESERVED.
Block Data to File
42
2014 LENOVO. ALL RIGHTS RESERVED.

Users work at the file level (ie. Word files,
excel spreadsheets. etc) and then the
applications we use change these to block
level data

Block level must go through the server first
Storage Model for SAN
Application
Server
Application
Server
Application
Server
Block-level access
Switch
Controllers
0110100101011101010001
• External storage
system connected
to SAN
• Controller-based
RAID
43
2014 LENOVO. ALL RIGHTS RESERVED.
Advantages of SAN




44
High-bandwidth capable of growing incrementally
Transfers very large blocks of data
Offers storage applications such as backup and
remote mirroring without bogging down LAN
Superior performance, reliability and flexible
connectivity
2014 LENOVO. ALL RIGHTS RESERVED.
SAN benefits
45
2014 LENOVO. ALL RIGHTS RESERVED.
Well-designed SAN
The major features that a well−designed SAN offers include:






46
High bandwidth—1Gbps to 10Gbps
Disaster recovery plans
Business continuity plan
Manageability
Easy integration
Lower total cost of ownership
2014 LENOVO. ALL RIGHTS RESERVED.
Key benefits of a Centralized SAN
 Infrastructure for easy storage connectivity and growth
 Easy management
 Performance
 “Freedom”
-Connecting everywhere every device into the same
storage network (SAN)
-On Line adding devices
47
2014 LENOVO. ALL RIGHTS RESERVED.
Fibre Channel Connectivity
48
2014 LENOVO. ALL RIGHTS RESERVED.
Logical Topologies
49
2014 LENOVO. ALL RIGHTS RESERVED.
Topology 1: Point – to - Point
50
2014 LENOVO. ALL RIGHTS RESERVED.
Point to Point
Controller
Computer
Memory
N_Port0
N_Port1
TX
RX
RX
TX
Bridge
Processor
I/O Bus
Transmit (TX)
Receive (RX)
51
2014 LENOVO. ALL RIGHTS RESERVED.
Drive
Enclosure
Topology 2 : Arbitrated LOOP
52
2014 LENOVO. ALL RIGHTS RESERVED.
Fibre Channel Arbitrated Loop
127 Nodes Maximum, Typical 5-30
NL_port 1
NL_port 0
TX
RX
RX
TX
NL_port 2
NL_port 3
TX
RX
RX
TX
Fibre Channel-AL
53
2014 LENOVO. ALL RIGHTS RESERVED.
Fibre Channel Arbitrated Loop
NL_port 0
NL_port 1
TX
RX
RX
TX
HUB
NL_port 3
TX
RX
RX
TX
127 Nodes Maximum
54
NL_port 2
2014 LENOVO. ALL RIGHTS RESERVED.
Fibre Channel Arbitrated Loop
NL_port 0
By pass defective or unused ports
TX
RX
RX
TX
NL_port 2
NL_port 3
TX
RX
RX
HUB
127 Nodes Maximum
55
NL_port 1
2014 LENOVO. ALL RIGHTS RESERVED.
TX
Topology 3: Switched Fabric
56
2014 LENOVO. ALL RIGHTS RESERVED.
Switched Fabric
N_port 0
N_port 3
TX
F_port A
RX
N_port 1
F_port D
N_port 2
RX
2014 LENOVO. ALL RIGHTS RESERVED.
RX
F_port B
TX
57
F_port E
Fabric
TX
RX
TX
F_port C
N_port 4
TX
RX
224 = 16 million nodes possible
Fully Redundant SAN
Server 1
HBA
Fibre Channel
switch
subsystem
Ethernet hub or switch
HBA
A
Server 2
B
HBA
HBA
Ethernet to client workstations
58
2014 LENOVO. ALL RIGHTS RESERVED.
Adding Capacity
Server 1
HBA
Fibre Channel
switch
subsystem A
A
HBA
B
Server 2
subsystem B
HBA
HBA
A
B
59
2014 LENOVO. ALL RIGHTS RESERVED.
A switched SAN
60
2014 LENOVO. ALL RIGHTS RESERVED.
SAN Model
61
2014 LENOVO. ALL RIGHTS RESERVED.
SAN Infrastructure
62
2014 LENOVO. ALL RIGHTS RESERVED.
SAN Components
SAN
Network
63
2014 LENOVO. ALL RIGHTS RESERVED.
SAN Components
SAN Servers
The server infrastructure is the reason for all SAN solutions. This infrastructure includes

a mix of server platforms such as Windows, UNIX (and its various flavors), and z/OS.

With initiatives such as server consolidation and e−business, the need for SANs will
increase, making the importance of storage in the network greater.
SAN Storage



The storage infrastructure is the foundation on which information relies, and therefore
must support a company's business objectives and business model. In this
environment simply deploying more and faster storage devices is not enough.
A SAN infrastructure provides enhanced network availability, data accessibility, and
system manageability.
It is important to remember that a good SAN begins with a good design.
SAN Interconnects


64
The first element that must be considered in any SAN implementation is the
connectivity of storage and server components typically using Fibre Channel.
It uses special connectivity devices
2014 LENOVO. ALL RIGHTS RESERVED.
SAN operability
65
2014 LENOVO. ALL RIGHTS RESERVED.
Fibre Channel layers
66
2014 LENOVO. ALL RIGHTS RESERVED.
SAN Architecture
67
2014 LENOVO. ALL RIGHTS RESERVED.
Fabric Channel layers
Lower Layers



FC−1 defines encoding schemes. These are used to synchronize
data for transmission.
FC−2 defines the framing protocol and flow control. This protocol
is self−configuring and
supports point−to−point, arbitrated loop, and switched topologies.
Upper Layers
Fibre Channel is a transport service that moves data quickly and
reliably between nodes. The two
upper layers enhance the functionality of Fibre Channel and provide
common implementations for interoperability.


68
FC−3 defines common services for nodes. One defined service is
multicast, to deliver one transmission to multiple destinations.
FC−4 defines upper layer protocol mapping. Protocols such as
FCP (SCSI), FICON, and IP can be mapped to the Fibre Channel
transport service.
2014 LENOVO. ALL RIGHTS RESERVED.
SAN High Level
69
2014 LENOVO. ALL RIGHTS RESERVED.
This is the storage area network:
70
2014 LENOVO. ALL RIGHTS RESERVED.
Zoning

It could be considered as a security feature and not just for
separating environments.

Zoning could also be used for test and maintenance purposes.

Zoning also introduces the flexibility to manage a switched fabric to
meet different user group objectives.

Zoning can be implemented in two ways:
-Hardware zoning
-Software zoning
71
2014 LENOVO. ALL RIGHTS RESERVED.
Zoning

72
Hardware zoning is based on the physical fabric port number
2014 LENOVO. ALL RIGHTS RESERVED.
Zoning

73
Software zoning is implemented by the fabric operating systems within
the fabric switches. When using software zoning, the members of the
zone can be defined using their WWN and WWPN.
2014 LENOVO. ALL RIGHTS RESERVED.
ISL Trunking


74
Trunking is a feature of switches that enables traffic to be
distributed across available inter−switch
links (ISLs) while still preserving in−order delivery.
2014 LENOVO. ALL RIGHTS RESERVED.
Fiber Channel types of ports
75

E_Port: This is an expansion port. A port is designated an E_Port when it is
used as an inter−switch expansion port (ISL) to connect to the E_Port of
another switch, to enlarge the switch fabric.

F_Port: This is a fabric port that is not loop capable. It is used to connect
an N_Port point−point to a switch.

FL_Port: This is a fabric port that is loop capable. It is used to connect an
NL_Port to the switch in a public loop configuration.

G_Port: This is a generic port that can operate as either an E_Port or an
F_Port. A port is defined as a G_Port after it is connected but has not
received a response to loop initialization or has not yet completed the link
initialization procedure with the adjacent Fibre Channel device.
2014 LENOVO. ALL RIGHTS RESERVED.
Fiber Channel types of ports
76


L_Port: This is a loop−capable node or switch port.
U_Port: This is a universal port—a more generic switch port than a G_Port.
It can operate as either an E_Port, F_Port, or FL_Port. A port is defined as
a U_Port when it is not connectedor has not yet assumed a specific
function in the fabric.

N_Port: This is a node port that is not loop capable. It is used to connect
an equipment port to the fabric.

NL_Port: This is a node port that is loop capable. It is used to connect an
equipment port to the fabric in a loop configuration through an L_Port or
FL_Port.
2014 LENOVO. ALL RIGHTS RESERVED.
Types or ports
77
2014 LENOVO. ALL RIGHTS RESERVED.