iSCSI Review
Introduction
With the release of the Fibre Channel and SAN based on it the storage world
staked on a network access to storage devices. Almost everyone announced
that the future belonged to the storage area networks. For several years
the FC interface was the only standard for such networks but today many
realize that it's not so. The SAN based on the FC has some disadvantages,
which are the price and difficulties of access to remote devices. At present
there are some initiatives which are being standardized; they are meant
to solve or diminish the problems. The most interesting of them is iSCSI.
The word iSCSI can often be seen in newspapers and ads of leading storage
device makers. However, different sources have very different views, and
some consider the iSCSI an indisputable leader for data storage systems
in the near future, others have already given it up for lost yet before
it was born.
Sun in opposition to IP Storage
Sun got into opposition to IP Storage. Sun Microsystems won't produce storage
area devices with an IP access. Mark Canepa, Vice-President of Sun, who
is in charge of production of all storage area systems said that IP Storage
was just a dream, reports Byte and Switch.
"Canepa said TCP/IP is impractical for use in storage area networks...
because the processing involved reading and forwarding packets appropriately
introduces lag time or latency. Canepa asserted that if storage over IP
does have a future, it will be a long time coming three to five years or
maybe never. Storage cannot run on top of a general-purpose stack designed
for everything; storage traffic has specific needs. The dream of slapping
together a TCP/IP network because everyone knows how easy it is to do it,
won't work. The technology hurdles are greater than people realize. Sun
is betting that Fibre Channel will be the main driver for SAN communication
for some time to come." So far nobody among storage system makers took
a so definite stand against IP Storage. Competitors of Sun - Hewlett-Packard
and IBM more or less support these technologies.
HP promises iSCSI support
"The final version of the new technology must appear in the first quarter
of 2002, HP will begin shipping a variety of storage products centered
around the emerging iSCSI (Internet Small Computer System Interface) standard
for linking data storage systems over Internet Protocol (IP), according
to Mark Thompson, worldwide marketing manager for HP's networked storage
division."
HP admits that users of Fibre Channel systems feel quite comfortable and have
a stronger bent for the modernized technology of FCIP than for iSCSI. At the same
time HP believes that the experience of working with solutions based on the IP
protocol, especially with the Ethernet, will make iSCSI solutions more attractive
for many customers.
IBM releases an iSCSI based product
The IBM TotalStorage™ IP Storage 200i provides storage that is directly
attachable to an Ethernet LAN. This high-performance storage product implements
newly emerging industry standards for the transport of SCSI storage protocols
over IP, commonly referred to as iSCSI.
Well, let's make go into the details ourselves and establish our own pros
and cons.
iSCSI
iSCSI (Internet Small Computer System Interface) is a TCP/IP-based
protocol for establishing and managing connections between IP-based storage
devices, hosts and clients.
iSCSI describes:
-
Transport protocol for SCSI which operates on top of TCP
-
New mechanism for encapsulating SCSI commands on an IP network
-
Protocol for a new generation of data storage systems that natively use
TCP/IP
But it's known that rules of packet delivery differ for IP and SCSI. In
IP packets are delivered without a strict order, it is also in charge of
data recovery, which takes more resources. At the same time, in SCSI, as
a channel interface, all packets must be delivered one after another without
delay, and breach of the order may result in data losses. In spite of the
fact that this problem, according to some experts, brings ambiguity into
practical use of the iSCSI technology, today there are devices that prove
its viability. The engineers developing the iSCSI managed to solve this
problem to some degree. The specification of the iSCSI requires a longer
packet's head. The head includes additional information which speeds up
packet
assembling by a great margin.
According to a senior system engineer of a Utah's university, the only
obstacle for popularization of the Ethernet as a base technology for establishing
storage area networks is a relatively great latency (close to 75 microseconds)
because of the peculiarities of the TCP/²Š stack. It can be a crucial problem
in High-End systems in case of simultaneous access to thousands of files.
Experts working on iSCSI address the problem of latency with a careful
attention. And although there are a lot of means developed to reduce influence
of parameters which cause delays in processing of IP packets, the iSCSI
technology is positioned for middle-level systems.
iSCSI develops quite rapidly. The need in a new standard was so strong
that during 14 months after the proposal on the iSCSI by IETF in February
2000 we got a lot of devices demonstrating capabilities of their interaction.
The Draft 0 on iSCSI published in July 2000 initiated realization of the
technology. In January 2001 IP Storage Forum was created within the SNIA
(Storage Networking Industry Association) which had already 50 members
in half a year; and a product released in April 2000 soon won the Enterprise
Networking Product Prize.
So, what is so attractive in the iSCSI for majors of the IT industry
who do not even consider contradictions of this standard.
Here are the most important applications and functions which can be
realized with data storage systems used:
Within the frames of those which can effectively be realized using modern
methods:
· Consolidation of data storage systems · Data backup · Server clusterization
· Replication · Recovery in emergency conditions
Here are new capabilities which can effectively be realized with the IP
Storage:
· SAN geographic distribution · QoS · Safety
In addition, new storage area systems with the iSCSI being native for them
provide more advantages:
· A single technology for connection of storage systems, servers and
clients within LAN, WAN, SAN · Great experience of industry in Ethernet
and SCSI technologies · Possibility of substantial geographic remoteness
of storage systems · Possibility to use management means for TCP/IP networks
To transfer data to storage devices with the iSCSI interface it's possible
to use not only data carriers, communicators and routers of existent LAN/WAN
but also usual network cards on the client's side. But it is followed by
considerable expenses of processor power on the client's side which uses
such card. According to the developers, the software iSCSI realization
can reach data rates of Gigabit Ethernet at a significant, about 100%,
CPU load. That is why it is recommended using special network cards which
support mechanisms of CPU unload before TCP stack processing. At present
(June 2002), such cards are produced by Intel.
The Intel PRO/1000T IP Storage Adapter (http://www.intel.ru/ru/network/connectivity/products/iscsi/index.htm)
is offered at 700USD. It contains a powerful Xscale processor, 32M memory
and transfers calculations related with iSCSI and TCP/IP and calculations
of checksums of TCP, IP frames to the integrated processor. According to
the company it can be as efficient as 500Mbit/s at 3-5% CPU load of a host
system.
iSCSI under a microscope
Fig. 1. IP network with iSCSI devices used
Here, each server, workstation and storage device support the Ethernet
interface and a stack of the iSCSI protocol. IP routers and Ethernet switches
are used for network connections.
The SAN makes possible to use the SCSI protocol in network infrastructures,
thus, providing high-speed data transfer at the block level between multiple
elements of data storage networks.
The Internet Small Computer System Interface also provides a block data
access, but over TCP/IP networks.
An architecture of a pure SCSI is based on the client/server model.
A client, for example, server or workstation, initiates requests for data
reading or recording from a target - server, for example, a data storage
system. Commands which are sent by the client and processed by the server
are put into the Command Descriptor Block (CDB). The server executes a
command which completion is indicated by a special signal alert. Encapsulation
and reliable delivery of CDB transactions between initiators and targets
through the TCP/IP network is the main function of the iSCSI, which is
due to be implemented in the medium untypical of SCSI, potentially unreliable
medium of IP networks.
Below is a model of the iSCSI protocol levels which allows us to get
an idea of an encapsulation order of SCSI commands for their delivery through
a physical carrier.
Fig. 2. Model of lower levels of the iSCSI protocol |
The iSCSI protocol controls data block transfer and confirms that I/O
operations are truly completed. In its turn, it is provided via one or
several TCP connections.
The iSCSI has four components:
-
iSCSI Address and Naming Conventions.
-
iSCSI Session Management.
-
iSCSI Error Handling.
-
iSCSI Security.
Address and Naming Conventions
As the iSCSI devices are participants of an IP network they have individual
Network Entities. Such Network Entity can have one or several iSCSI nodes.
Fig. 3. Model of Network Entities |
An iSCSI node is an identifier of SCSI devices (in a network entity)
available through the network. Each iSCSI node has a unique iSCSI name
(up to 255 bytes) which is formed according to the rules adopted for Internet
nodes. For example, fqn.com.ustar.storage.itdepartment.161. Such name has
an easy-to-perceive form and can be processed by the Domain Name System
(DNS). An iSCSI name provides a correct identification of an iSCSI device
irrespective of its physical location. At the same time in course of handling
data transfer between devices it's more convenient to use a combination
of an IP address and a TCP port which are provided by a Network Portal.
The iSCSI protocol together with iSCSI names provides a support for aliases
which are reflected in the administration systems for better identification
and management by system administrators.
Session Management
The iSCSI session consists of a Login Phase and a Full Feature Phase which
is completed with a special command.
The Login Phase of the iSCSI is identical to the Fibre Channel Port
Login process (PLOGI). It is used to adjust various parameters between
two network entities and confirm an access right of an initiator. If the
iSCSI Login Phase is completed successfully the target confirms the login
for the initiator; otherwise, the login is not confirmed and a TCP connection
breaks.
As soon as the login is confirmed the iSCSI session turns to the FULL
Feature Phase. If more than one TCP connection was established the iSCSI
requires that each command/response pair goes through one TCP connection.
Thus, each separate read or write command will be carried out without a
necessity to trace each request for passing different flows. However, different
transactions can be delivered through different TCP connections within
one session.
Fig. 4. iSCSI Write example |
At the end of a transaction the initiator sends/receives last data and
the target sends a response which confirms that data are transferred successfully.
The iSCSI logout command is used to complete a session - it delivers
information on reasons of its completion. It can also send information
on what connection should be interrupted in case of a connection error,
in order to close troublesome TCP connections.
Error Handling
Because of a high probability of errors in data delivery in some IP networks,
especially WAN, where the iSCSI can work, the protocol provides a great
deal of measures for handling errors.
So that error handling and recovery can work correctly both the initiator
and the target must be able to buffer commands before they are confirmed.
Each terminal must have a possibility to recover selectively a lost or
damaged PDU within a transaction for recovery of data transfer.
Here is the hierarchy of the error handling and recovery after failures
in the iSCSI:
-
The lowest level - identification of an error and data recovery on the
SCSI task level, for example, repeated transfer of a lost or damaged PDU.
-
Next level - a TCP connection which transfers a SCSI task can have errors.
In this case there is an attempt to recover the connection.
-
At last, the iSCSI session can be damaged. Termination and recovery of
a session are usually not required if recovery is implemented correctly
on other levels, but the opposite can happen. Such situation requires that
all TCP connections be closed, all tasks, underfulfilled SCSI commands
be completed, and the session be restarted via the repeated login.
Security
As the iSCSI can be used in networks where data can be accessed illegally,
the specification allows fpr different security methods. Such encoding
means as IPSec which use lower levels do not require additional matching
because they are transparent for higher levels, and for the iSCSI as well.
Various solutions can be used for authentication, for example, Kerberos
or Private Keys Exchange, an iSNS server can be used as a repository of
keys.
Others (iFCP, FCIP)
The IP Storage (IPS) work group was created within the frames of developing
network storage technologies in the Internet Engineering Task Force (IETF);
it has the following directions:
-
iSCSI (Internet Small Computer Systems Interface)
-
FCIP (Fibre Channel over TCP/IP)
-
iFCP (Internet Fibre Channel Protocol)
-
iSNS (Internet Storage Name Service)
In January 2001 IP Storage Forum was established within SNIA (Storage Networking
Industry Association). Today the Forum includes three subgroups: FCIP,
iFCP, iSCSI, each representing a protocol which is under the IETF protection.
FCIP - a tunnel protocol based on the TCP/IP which is designed
for connection of geographically far FC SANs without affecting FC and IP
protocols.
iFCP - TCP/IP based protocol for connection of FC data storage
systems using the IP infrastructure together or instead of FC switching
and routing elements.
iSCSI - described above...
For better understanding of positioning of these three protocols there
is a diagram of networks based on them.
Fig. 5. Diagram of IP Storage networks
Fibre Channel over IP
The most revolutionary protocol among these three is Fibre Channel over
IP. It doesn't bring in any changes into the SAN structure and organization
of storage area systems. The main idea of this protocol is to make functional
integration of geographically remote storage networks.
Here is the stack of the FCIP protocol:
Fig. 6. Lower levels of the FCIP protocol |
FCIP helps to effectively solve a problem of geographical distribution,
and integration of SANs on large distances. This protocol is entirely transparent
for existent FC SANs and involves usage of infrastructure of modern MAN/WAN
networks. So, if you want to merge geographically remote FC SANs with new
functionality enabled you will have to get just one FCIP gateway and connection
to MAN/WAN networks. A geographically distributed SAN based on the FCIP
is taken by SAN devices as a usual FC network, and it is seen as a usual
IP traffic for a MAN/WAN network it is connected to.
FCIP IETF IPS Working Group Draft Standard specifies:
-
rules of encapsulation of FC frames for delivery through TCP/IP;
-
rules of using encapsulation for creation of a virtual connection between
FC devices and elements of an FC network;
-
TCP/IP environment for support of creation of a virtual connection and
support of FC traffic tunneling through an IP network including safety,
integrity of data and a data rate issue.
Here are some applied problems which can be successfully solved using the
FCIP protocol: remote backup, data recovery and a shared data access. With
high-speed MAN/WAN communications one can also use synchronous data doubling
and a shared distributed access to data storage systems.
iFCP
Internet Fibre Channel Protocol is a protocol which provides FC traffic
delivery over the TCP/IP transport between iFCP gateways. In this protocol
an FC transport level is replaced with a transport of the IP network, the
traffic between FC devices is routed and switched by the means of TCP/IP.
The iFCP protocol allows connecting current FC data storage systems to
an IP network with a support of network services which are necessary for
these devices.
Here is how an iFCP protocol stack looks like:
Fig. 7. Lower levels of the iFCP protocol |
According to the specification iFCP:
-
overlays FC frames for their delivery to a predetermined TCP connection;
-
FC services of message delivery and routing are overlapped in the iFCP
gateway device; therefore, network structures and components of the FC
do not mix in one FC SAN but are managed by the TCP/IP means;
-
dynamically creates IP tunnels for FC frames
An important feature of the iFCP is that this protocol provides an FC device-to-device
connection via an IP network which is a more flexible scheme in comparison
to the SAN-to-SAN. For example, if the iFCP has a TCP connection between
pairs of N_Ports of two FC devices such connection can have its own QoS
level which will be different from a QoS level of another pair of FC devices.
Conclusion
I'm quite sure that in the near future the Fibre Channel won't disappear
and the FC SAN market will be further developing. At the same time the
IP Storage protocols will make possible to use effectively storage area
networks in those applications for which the FC can't provide an effective
realization. With the FCIP and iFCP protocols data storage networks will
be geographically distributed. And the iSCSI will make possible to use
advantages of the SAN in the spheres which are still not or ineffectively
realized within popular technologies.
P.S.
The rapid development of data storage networks is what the conception of
the World Wide Storage Area Network based on. WWSAN provides for an infrastructure
which will support a high-speed access and storage of data distributed
all over the world. The conception is very close to the WWW but is based
on different services. One of examples is servicing a manager who travels
around the world with presentations. WWWSAN provides for transparent transfer
of "mobile" data according to how their owner travels all around the world.
Therefore, wherever such manager can be, he will always have a high-speed
access to the data he needs, and an operation with them won't require a
complicated ineffective synchronization via the WWW.
The conception of building the World Wide Storage Area Network excellently
fits in the development of modern IP Storage technologies.
Terms and abbreviations:
-
SAN - Storage Area Network
-
CDB - command descriptor block.
-
PDU - Protocol Data Unit.
-
QoS - Quality of Service (usually describes a network through latency and
band of a signal).
-
SNIA - Storage Networking Industry Association.
-
DNS - Domain Name Server.
-
PLOGI - Fibre Channel Port Login.
-
iSCSI - Internet Small Computer Systems Interface
-
FCIP - Fibre Channel over TCP/IP
-
iFCP - Internet Fibre Channel Protocol
-
iSNS - Internet Storage Name Service
-
WWSAN - World Wide Storage Area Network
Literature:
-
SAN after Fibre Channel, - Lucas Mearian
-
IP Storage Tutorial, - SNIA, http://www.snia.org/
-
iSCSI Technical White Paper, - SNIA, http://www.snia.org/
-
Internet Fibre Channel Protocol (iFCP) - A Technical Overview, - SNIA,
http://www.snia.org/
-
Storage Forum, - HP EMEA, June 12-13, 2002.
