Best practices for fiber channel host connectivity

Follow

Cabling and zoning

To achieve the best performance from InfiniBox and highest availability for hosts, zoning each host to all three (3) active-active-active Nodes on the system is strongly recommended. By doing this, we ensure balanced utilization of all the resources on the system in an optimal way. However, in some environments there may be too many hosts or insufficient FC switch ports available, so tradeoffs may have to be made. The InfiniBox will still make a best effort of utilizing and balancing all the Nodes, even if a given host sends I/O to just one or two Nodes by using the Infiniband interconnect between the nodes.
Generally speaking, the following guidelines should be observed when zoning hosts to an InfiniBox.

  • Each physical host should be zoned to all 3 storage nodes via at least 2 independent HBA ports (initiators) on 2 independent SAN fabrics.
  • A maximum 1-to-3 fan-out from host (initiator) to storage (target) port should normally be used. Thus for a host with two HBA ports, there will be 2 x 3 = 6 paths per LUN. For a host with 4 HBA ports, there will be 4 x 3 = 12 paths per LUN.
  • Zones should be created as single-initiator WWPN zones. Thus each zone will normally have a maximum of 4 WWPNs assigned: 1 WWPN from the host (initiator) and 3 from the storage (target).
  • As discussed in the previous section, hosts should only use supported and properly configured multipathing software to ensure proper load balancing across the storage nodes and maximum performance.
  • The table below shows examples of common host port configurations with the maximum theoretical BW achievable for either read or write. Performance for any individual host will depend heavily on the actual host I/O profile.

Bandwidth / fan-out table

Host Port Speed# Host Ports Recommended Fan-out Paths Per LUN Peak Theoretical Bandwidth
4GB21:3 6 800 MB/s
4GB41:3 12 1.6 GB/s
4GB61:1 or 1:2 6 or 122.4 GB/s
4GB121:1 12 4.8 GB/s
8GB21:3 6 1.6 GB/s
8GB41:3 12 3.2 GB/s
8GB61:1 or 1:26 or 124.8 GB/s
8GB121:1 12 9.6 GB/s

Physical cabling

To achieve the recommended zoning goals, the physical cabling between the SAN fabrics and InfiniBox needs to be performed in a manner consistent with maximizing both performance and availability. This means cabling two separate SAN fabrics to each InfiniBox target HBA with as many connections as necessary to support the host infrastructure. Typical installations either use 12 of the InfiniBox FC ports or all 24 of them. Consider the diagram below as a basic guide for the physical cabling, where each green line represents 2-4 physical connections between the InfiniBox and the SAN fabrics.

As stated previously, when connecting InfiniBox to the Fiber Channel fabric, special care has to be taken to connecting ports from all three nodes and their HBAs to each switch. To avoid confusion, InfiniBox comes with a simple, color-coded patch panel that makes the cabling simpler. There are two types of patch panels, depending on the release.

Recommended configuration

The InfiniBox storage system has three Nodes, each has 8 fiber-channel ports.

It is recommended that each switch will be connected to each node with 2 or more FC:

  • One to the top row (first target HBA)
  • One to the bottom row (second target HBA) 

This will also allow each host to connect to two different target ports from 2 different target HBAs. Zoning should be done such that each initiator sees one port from each of the three nodes. 

  • That is, each initiator should have 1(initiator ports) * 3(nodes) * 2(target ports per node) = 6 paths.

Example

In the following example, the host has two initiators. Each initiator is connected to two ports on each node, 6 paths in total:

  • Fabric A is connected to:
    • Node 1 Port 5
    • Node 2 Port 5
    • Node 3 Port 5
  • Fabric B is connected to:
    • Node 1 Port 1
    • Node 2 Port 1
    • Node 3 Port 1

 It is recommended for every host to have two independent initiators, to avoid a single-point-of-failure on the host side.

 

 


Comments

Powered by Zendesk