Fibre Channel and FCoE SAN configuration details

Apply the following Storwize® V3500 configuration details for Fibre Channel and Fibre Channel Over Ethernet (FC/FCoE Gateway, FCF) switches to ensure that you have a valid configuration.

SAN fabric function is available only if a Fibre Channel option is installed. Otherwise, this information does not apply.

Configuring your SAN with at least two independent switches, or networks of switches, ensures a redundant fabric with no single point of failure. If one of the two SAN fabrics fails, the configuration is in a degraded mode, but is still valid. Maintain separate fabrics for FCoE and FC. If you attempt to combine these fabrics, you might risk adding more paths to the volumes. Supported configurations allow a maximum of eight paths. A SAN with only one fabric is a valid configuration but risks loss of access to data if the fabric fails. SANs with only one fabric are exposed to a single point of failure.

For Fibre Channel connections, the Storwize V3500 node canisters must be connected to either SAN switches or directly connected to a host port. When iSCSI hosts are attached to Storwize V3500 node canisters, Ethernet switches must be used.

All back-end storage systems must always be connected to SAN switches only. Multiple connections are permitted from redundant storage systems to improve data bandwidth performance. A connection between each redundant storage system and each counterpart SAN is not required. For example, in an IBM® DS4000® configuration in which the IBM DS4000 contains two redundant storage systems, only two storage system minihubs are usually used. Storage system A is connected to counterpart SAN A, and storage system B is connected to counterpart SAN B. Any configuration that uses a direct physical connection between the Storwize V3500 node and the storage system is not supported.

When you attach a node to a SAN fabric that contains core directors and edge switches, connect the node ports to the core directors and connect the host ports to the edge switches. In this type of fabric, the next priority for connection to the core directors is the storage systems, leaving the host ports connected to the edge switches.

A Storwize V3500 SAN must follow all switch manufacturer configuration rules, which might place restrictions on the configuration. Any configuration that does not follow switch manufacturer configuration rules is not supported.

Mixing manufacturer switches in a single SAN fabric

Within an individual SAN fabric, only mix switches from different vendors if the configuration is supported by the switch vendors. When you use this option for FCF Switch to FC Switch connectivity, review and plan as documented in ISL oversubscription.

Fibre Channel over Ethernet servers and Storwize V3500 connections to the existing Fibre Channel SAN

Fibre Channel over Ethernet (FCoE) is available on the Storwize V3500 only if a FCoE option is installed. Otherwise this information does not apply.

FCoE servers and Storwize V3500 systems can be connected in several different ways. The following examples show the various supported configurations.

Figure 1 shows an Storwize V3500 system connected to a Fibre Channel forwarder switch along with any FCoE hosts and FCoE storage systems. The connections are 10 GB Ethernet. The Fibre Channel forwarder is linked to the existing Fibre Channel SAN by using Fibre Channel ISLs. Any Fibre Channel hosts or storage systems remain on the existing Fibre Channel SAN. The connection to the Storwize V3500 can be via the SAN (if the Storwize V3500 is connected via Fibre Channel) or via the Fibre Channel forwarder switch to the FCoE ports on the Storwize V3500 system.

Figure 1. Fibre Channel forwarder linked to existing Fibre Channel SAN
This figure depicts a FC forwarder linked to an existing FC SAN.

The second example, Figure 2, is almost the same as the first example but without an existing Fibre Channel SAN. It shows a Storwize V3500 system that is connected to a Fibre Channel forwarder switch along with any FCoE hosts and FCoE storage systems. The connections are 10 GB Ethernet.

Figure 2. Fibre Channel forwarder linked to hosts and storage systems without an existing Fibre Channel SAN
This figure depicts a FC forwarder linked to hosts and storage systems without an existing FC SAN.

In the third example, Figure 3, a Fibre Channel host connects into the Fibre Channel ports on the Fibre Channel forwarder. The Storwize V3500 system is connected to a Fibre Channel forwarder switch along with any FCoE storage systems. The connections are 10 GB Ethernet. The Fibre Channel forwarder is linked to the existing Fibre Channel SAN by using Fibre Channel ISLs. Any Fibre Channel hosts or storage systems remain on the existing Fibre Channel SAN. The FCoE host connects to a 10 GB Ethernet switch (transit switch) that is connected to the Fibre Channel forwarder.

Figure 3. Fibre Channel host connects into the Fibre Channel ports on the Fibre Channel forwarder
This figure depicts a FC host that connects into the Fibre Channel ports on the Fibre Channel forwarder.

The fourth example, Figure 4, is about the same as the previous example but without an existing Fibre Channel SAN. The Fibre Channel hosts connect to Fibre Channel ports on the Fibre Channel forwarder.

Figure 4. Fibre Channel host connects into the Fibre Channel ports on the Fibre Channel forwarder without an existing Fibre Channel SAN.
This figure depicts a FC host connecting into the Fibre Channel ports without an existing FC SAN.

ISL oversubscription

Perform a thorough SAN design analysis to avoid ISL congestion. Do not configure the SAN to use Storwize V3500 to Storwize V3500 traffic or Storwize V3500 to storage system traffic across ISLs that are oversubscribed. For host to Storwize V3500 traffic, do not use an ISL oversubscription ratio that is greater than 7 to 1. Congestion on the ISLs can result in severe Storwize V3500 performance degradation and I/O errors on the host.

When you calculate oversubscription, you must account for the speed of the links. For example, if the ISLs run at 4 Gbps and the host runs at 2 Gbps, calculate the port oversubscription as 7 * (4/2). In this example, the oversubscription can be 14 ports for every ISL port.
Note: The Storwize V3500 port speed is not used in the oversubscription calculation.

ISL oversubscription rules apply to FCoE Switches.

Storwize V3500 in a SAN with director class switches

You can use director class switches within the SAN to connect large numbers of RAID controllers and hosts to a Storwize V3500 system. Because director class switches provide internal redundancy, one director class switch can replace a SAN that uses multiple switches. However, the director class switch provides only network redundancy; it does not protect against physical damage (for example, flood or fire), which might destroy the entire function. A tiered network of smaller switches or a core-edge topology with multiple switches in the core can provide comprehensive redundancy and more protection against physical damage for a network in a wide area. Do not use a single director class switch to provide more than one counterpart SAN because this configuration does not constitute true redundancy.