This document guides users on how to set up the Ethernet network for InfiniBox services. Three types of services are supported over Ethernet: NAS, iSCSI and replication. Normally, each service can use two or more ports per node, providing high availability and increased throughput.

Introduction

Scope of document

This document provides guidelines and instructions for setting up InfiniBox services. The document is built upon past releases of Best Practices and Deployment Considerations documents that are still available on the INFINIDAT Support website. If you are running an InfiniBox version older that 4.0, this document complements but does not entirely replace past documentation.

Obsolete documentation

• Deployment consideration for setting up services - User documentation for InfiniBox v4.0 and former InfiniBox releases, includes an article named "Deployment Considerations for Setting up Services". This article is now obsolete. The instructions provided in the article have been merged into this document

Deploying InfiniBox Services

InfiniBox Services

InfiniBox supports three types of services over Ethernet networks: NAS, iSCSI and replication. Normally, each service can use one, two or more ports per node, which providing high availability and increased throughput.
NAS and replication services can easily share the ports (in pairs) and this is a common configuration. It is neither customary nor recommended to share ports between iSCSI and other services due to the differing nature of the relevant protocols:

• The iSCSI protocol has a built-in mechanism to ensure high-availability and high-performance using multipathing. This is the standard option for iSCSI deployment and is the recommended configuration for using iSCSI with InfiniBox.
• NAS and replication services do not have such built-in support, so LACP (link aggregation) is recommended to provide high-availability and increased performance.

The following table summarizes the different aspects of iSCSI vs. NAS and replication services:

Selecting Ethernet Ports for the Services

Ports should be grouped into pairs so they can be allocated to services. For example, assign ETH1 and ETH2 as one pair, with ETH3 and ETH4 as another pair. Analyze which services are required, and whether is there a need for physical separation between them. Consider the recommendation to isolate iSCSI and the fact that there are a limited number of pairs (depending on InfiniBox model).

Switch requirements

It is also recommended to set the ports connected to InfiniBox to account for changes using a spanning-tree algorithm. For many network vendor implementations, this will be called “portfast”, “edge” or “edge-port” in the switch configuration semantics.

InfiniBox Fx1xx

InfiniBox Fx1xx models have two Ethernet data ports per node (total of 6 ports per system). 

Use the following table to decide which port groups you need to create, and how many Ethernet ports to place in each port group:

InfiniBox Fx2xx: four Ethernet Data Ports Per Node

InfiniBox Fx2xx models have four Ethernet data ports per node (total of 12 ports per system). 

Use the following table to decide which port groups you need to create, and how many Ethernet ports to place in each port group:

InfiniBox Fx3xx

InfiniBox Fx3xx models have six Ethernet data ports per node (total of 18 ports per system). 

Use the following table to decide which port groups you need to create, and how many Ethernet ports to place in each port group:

Connecting the Switch to the InfiniBox Patch Panel

iSCSI deployment

1. Connect ports to the switches.

   1. Select one port from each node (ETH1 in the above example), and connect it to iSCSI Switch A

   2. Select one port from each node (ETH2 in the above example), and connect it to iSCSI Switch B
      
      

2. The following steps highlight the relevant configuration for iSCSI. For more details, see: Creating an iSCSI service in this document.

   1. Create a network space and call it iSCSI-A:

      1. Create a new Port Group with ETH1 port from every node
      2. Provide 6 IP addresses for the Network Space iSCSI-A 
   2. Create a network space and call it iSCSI-B:
      1. Create a new Port Group with ETH2 port from every node
      2. Provide 6 IP addresses for the Network Space iSCSI-B

NAS and/or Replication Deployment

1. Use two switches that support creating LACP LAGs that spread across the switches.

   Typically, stacked switches support such a configuration.

   Some switch chassis also support this (e.g. Cisco Nexus Virtual PortChannel).

2. Select one port from each node (ETH3 in the above example), and connect it to switch module 1
3. Select one port from each node (ETH4 in the above example), and connect it to switch module 2
   
   
   
   
4. Configure a LAG for each couple of ports, as indicated above.

5. The following steps highlight the relevant configuration for NAS and Replication. For more details, see: Creating a NAS Service

   Create a network space:

   1. Create a new Port Group with ETH3 and ETH4 ports from every node
   2. Provide IP addresses for the Network Space:
      • For NAS service, 6 IPs are required
      • For replication service, more than 6 IP addresses are required (see IP addresses in the section InfiniBox Best Practices Guide for Setting Up the Replication Service)
6. If you need to run both NAS and replication on the same Port Group, create another Network Space using the same Port Group.

Performance and Redundancy

Performance 

Specific details about the performance of NAS can be found in InfiniBox Best Practices Guide for Setting Up a NAS Service.

Specific details about the performance of replication can be found in InfiniBox Best Practices Guide for Setting up the Replication Service.

Ensuring Redundancy

"Good Path" ensures redundancy to all virtual IPs (NAS/Replication).
For iSCSI 6 IPs are also virtually applied - however each connection is independently managed with MPIO on the client.

Ensuring Redundancy in the Case of a Failed NIC

1 Failed NIC maintains access to all Virtual IPs although one controller has limited maximum bandwidth.

Ensuring Redundancy in the Case of a Failed Node

Access to all IPs is maintained as IPs are redistributed across the remaining controllers. Bandwidth is limited to the aggregate maximum bandwidth of the remaining controller ports.

Ensuring Redundancy in the Case of a Failed Switch

Access to all virtual IPs is maintained although maximum aggregate performance is reduced to 50%. Typically this has a low performance impact as clients generally do not utilize all of the available bandwidth.

This document guides users on how to set up the Ethernet network for InfiniBox services. Three types of services are supported over Ethernet: NAS, iSCSI and replication. Normally, each service can use two or more ports per node, providing high availability and increased throughput.

Introduction

Scope of document

This document provides guidelines and instructions for setting up InfiniBox services. The document is built upon past releases of Best Practices and Deployment Considerations documents that are still available on the INFINIDAT Support website. If you are running an InfiniBox version older that 4.0, this document complements but does not entirely replace past documentation.

Obsolete documentation

• Deployment consideration for setting up services - User documentation for InfiniBox v4.0 and former InfiniBox releases, includes an article named "Deployment Considerations for Setting up Services". This article is now obsolete. The instructions provided in the article have been merged into this document

Deploying InfiniBox Services

InfiniBox Services

InfiniBox supports three types of services over Ethernet networks: NAS, iSCSI and replication. Normally, each service can use one, two or more ports per node, which providing high availability and increased throughput.
NAS and replication services can easily share the ports (in pairs) and this is a common configuration. It is neither customary nor recommended to share ports between iSCSI and other services due to the differing nature of the relevant protocols:

• The iSCSI protocol has a built-in mechanism to ensure high-availability and high-performance using multipathing. This is the standard option for iSCSI deployment and is the recommended configuration for using iSCSI with InfiniBox.
• NAS and replication services do not have such built-in support, so LACP (link aggregation) is recommended to provide high-availability and increased performance.

The following table summarizes the different aspects of iSCSI vs. NAS and replication services:

Selecting Ethernet Ports for the Services

Ports should be grouped into pairs so they can be allocated to services. For example, assign ETH1 and ETH2 as one pair, with ETH3 and ETH4 as another pair. Analyze which services are required, and whether is there a need for physical separation between them. Consider the recommendation to isolate iSCSI and the fact that there are a limited number of pairs (depending on InfiniBox model).

Switch requirements

It is also recommended to set the ports connected to InfiniBox to account for changes using a spanning-tree algorithm. For many network vendor implementations, this will be called “portfast”, “edge” or “edge-port” in the switch configuration semantics.

InfiniBox Fx1xx

InfiniBox Fx1xx models have two Ethernet data ports per node (total of 6 ports per system). 

Use the following table to decide which port groups you need to create, and how many Ethernet ports to place in each port group:

InfiniBox Fx2xx: four Ethernet Data Ports Per Node

InfiniBox Fx2xx models have four Ethernet data ports per node (total of 12 ports per system). 

Use the following table to decide which port groups you need to create, and how many Ethernet ports to place in each port group:

InfiniBox Fx3xx

InfiniBox Fx3xx models have six Ethernet data ports per node (total of 18 ports per system). 

Use the following table to decide which port groups you need to create, and how many Ethernet ports to place in each port group:

Connecting the Switch to the InfiniBox Patch Panel

iSCSI deployment

1. Connect ports to the switches.

   1. Select one port from each node (ETH1 in the above example), and connect it to iSCSI Switch A

   2. Select one port from each node (ETH2 in the above example), and connect it to iSCSI Switch B
      
      

2. The following steps highlight the relevant configuration for iSCSI. For more details, see: Creating an iSCSI service in this document.

   1. Create a network space and call it iSCSI-A:

      1. Create a new Port Group with ETH1 port from every node
      2. Provide 6 IP addresses for the Network Space iSCSI-A 
   2. Create a network space and call it iSCSI-B:
      1. Create a new Port Group with ETH2 port from every node
      2. Provide 6 IP addresses for the Network Space iSCSI-B

NAS and/or Replication Deployment

1. Use two switches that support creating LACP LAGs that spread across the switches.

   Typically, stacked switches support such a configuration.

   Some switch chassis also support this (e.g. Cisco Nexus Virtual PortChannel).

2. Select one port from each node (ETH3 in the above example), and connect it to switch module 1
3. Select one port from each node (ETH4 in the above example), and connect it to switch module 2
   
   
   
   
4. Configure a LAG for each couple of ports, as indicated above.

5. The following steps highlight the relevant configuration for NAS and Replication. For more details, see: Creating a NAS Service

   Create a network space:

   1. Create a new Port Group with ETH3 and ETH4 ports from every node
   2. Provide IP addresses for the Network Space:
      • For NAS service, 6 IPs are required
      • For replication service, more than 6 IP addresses are required (see IP addresses in the section InfiniBox Best Practices Guide for Setting Up the Replication Service)
6. If you need to run both NAS and replication on the same Port Group, create another Network Space using the same Port Group.

Performance and Redundancy

Performance 

Specific details about the performance of NAS can be found in InfiniBox Best Practices Guide for Setting Up a NAS Service.

Specific details about the performance of replication can be found in InfiniBox Best Practices Guide for Setting up the Replication Service.

Ensuring Redundancy

"Good Path" ensures redundancy to all virtual IPs (NAS/Replication).
For iSCSI 6 IPs are also virtually applied - however each connection is independently managed with MPIO on the client.

Ensuring Redundancy in the Case of a Failed NIC

1 Failed NIC maintains access to all Virtual IPs although one controller has limited maximum bandwidth.

Ensuring Redundancy in the Case of a Failed Node

Access to all IPs is maintained as IPs are redistributed across the remaining controllers. Bandwidth is limited to the aggregate maximum bandwidth of the remaining controller ports.

Ensuring Redundancy in the Case of a Failed Switch

Access to all virtual IPs is maintained although maximum aggregate performance is reduced to 50%. Typically this has a low performance impact as clients generally do not utilize all of the available bandwidth.