Follow

Introduction

IBM® PowerVC™ is virtualization and cloud management software that is built on OpenStack.  

InfiniBox storage systems can be used in an OpenStack environment by utilizing the Infinidat Cinder or Manila drivers. IBM PowerVC allows any OpenStack supported storage system, including InfiniBox storage, to be used as a Pluggable Storage device, subject to IBM testing and validation which Infinidat has completed.

This document describes how to configure and use InfiniBox storage in a PowerVC environment, and contains the following sections:

  • Register an InfiniBox system with PowerVC
  • PowerVC Fabric Zoning for InfiniBox systems
  • Provision a new volume to a PowerVC VM
  • Migrate PowerVC volumes to InfiniBox

Supported InfiniBox Models

All InfiniBox models compatible with the current Infinidat Cinder driver are supported for use with the IBM PowerVC.

Supported Software

InfiniBox Software5.x and above
PowerVC Versions1.4 and above

The information presented in this document is based on testing performed with PowerVC version 2.0.1, which is based on OpenStack Ussuri.

Procedures may differ for other PowerVC versions.

Setup

Preparation

1. Create InfiniBox User for PowerVC

On the InfiniBox, create a pool and a user account for use with PowerVC. You can do this via the InfiniBox GUI, API, or CLI as shown below. Note that <InfiniBox-Pool-Name>, <InfiniBox-Pool-Capacity><InfiniBox-User-Name>, and <InfiniBox-User-Email> are placeholders for you to replace according to your preferred practices.

admin@InfiniBox> pool.create name=<InfiniBox-Pool-Name> physical_capacity=<InfiniBox-Pool-Capacity>
Pool "<InfiniBox-Pool-Name>" created
admin@InfiniBox> user.create name=<InfiniBox-User-Name> role=POOL_ADMIN email=<InfiniBox-User-Email>
Please enter a password for user "<InfiniBox-User-Name>":
Local user "<InfiniBox-User-Name>" created
admin@InfiniBox> pool.add_admin pool=<InfiniBox-Pool-Name> user=<InfiniBox-User-Name>
Granted provisioning privileges on pool <InfiniBox-Pool-Name> to pool admin: "local user "<InfiniBox-User-Name>""

2. Install Infinidat Python SDK

On the PowerVC management server/VM, use PyPi to install the Infinidat Python SDK:

# pip install infinisdk

3. Obtain InfiniBox Cinder Driver

In PowerVC, "Pluggable storage" refers to any OpenStack compatible storage array that has a vendor-distributed Cinder driver. InfiniBox is considered a pluggable storage array. 

Unlike typical OpenStack deployments, when the InfiniBox is configured as a storage provider for PowerVC, do not modify any of the Cinder configuration files directly. Instead, use the steps provided in this document.

On the PowerVC management server/VM, pull down the Cinder git repository and copy the Infinidat driver into the relevant folder:

  • For PowerVC 1.x: /usr/lib/python2.7/site-packages/cinder/volume/drivers
  • For PowerVC 2.x: /usr/lib/python3.6/site-packages/cinder/volume/drivers

Note that the Infinidat driver file permissions must at least be world-readable.

In the commands below, the stable/ussuri branch of OpenStack is used because PowerVC 2.0.1 is based on OpenStack Ussuri.

# git clone --branch stable/ussuri https://git.openstack.org/openstack/cinder.git
# cp cinder/cinder/volume/drivers/infinidat.py /usr/lib/python3.6/site-packages/cinder/volume/drivers/

Contact Infinidat for any infinidat.py driver patches that may not be available in upstream OpenStack repositories yet.

Register the InfiniBox system with PowerVC

1. Prepare PowerVC Properties File

Prepare a text file with the following contents. This file is temporary, and will not be needed after the system is registered with PowerVC. The following items shown in the example below are placeholders that should be filled based on your environment:

  • ibox-properties-file - this file name is arbitrary. Use a different file name for each InfiniBox system.
  • <InfiniBox-Floating-IP> - this is the floating IP address for managing the InfiniBox, found in the InfiniBox settings.
  • <InfiniBox-User-Name> and <InfiniBox-User-Password> - these are the InfiniBox credentials created for PowerVC management access in preparation step 1, earlier in this document.
  • <InfiniBox-Pool-Name> - this is the InfiniBox pool name that was created for PowerVC use in preparation step 1, earlier in this document.

To register a second InfiniBox system, repeat this process using an updated properties file.

# cat > ibox-properties-file
[backend_defaults]
san_ip = <InfiniBox-Floating-IP>
san_login = <InfiniBox-User-Name>
san_password = <InfiniBox-User-Password>
san_thin_provision = true
infinidat_pool_name = <InfiniBox-Pool-Name>
use_multipath_for_image_xfer = true
infinidat_storage_protocol = fc
Fibre Channel (fc) is the only protocol PowerVC supports for pluggable storage, as of PowerVC 2.0.1 (August 2021).

2. Register Provider with PowerVC

Run the powervc_register command using the properties file as follows. This command will run as root and will prompt for the root password. Note <InfiniBox-Provider-Name> is a placeholder for you to replace according to your preferred practices - e.g. you may wish to use the hostname of the InfiniBox system you are setting up.

# powervc-register -o add -r storage -d cinder.volume.drivers.infinidat.InfiniboxVolumeDriver -n <InfiniBox-Provider-Name> -p ibox-properties-file
Enter password for root:                                                                      

The Storage Provider '<InfiniBox-Provider-Name>' added to PowerVC.

PowerVC Fabric Zoning for InfiniBox systems

Fabric zoning is performed automatically by PowerVC as described at https://www.ibm.com/docs/en/powervc/2.0.1?topic=ps-how-powervc-sets-up-fibre-channel-switch-zoning, except if there is a need to deploy a VM using an existing Image. In that case the PowerVC Management Server/VM must be manually zoned to all three nodes of the InfiniBox on both fabrics. This is necessary because the InfiniBox doesn't currently support the concept of Volume Cloning, and to work around this, the InfiniBox Cinder volume driver creates a new volume, temporarily attaches it to the PowerVC server, copies the Image onto it, and then detaches it from the PowerVC server before making it available to the VM being deployed. This is the only portion of zoning that is manual. While deploying a VM to InfiniBox storage, or attaching InfiniBox volumes to an existing VM, PowerVC performs the fabric zoning necessary to connect that VM to all three InfiniBox nodes over two fabrics when configured as described in this document.

Given that the InfiniBox system can have up to 24 Fibre Channel ports connected into the fabrics, it is crucial to limit the number of ports used for creating zones in order to prevent a proliferation of paths used by a VM to access a volume. At the same time, it is also crucial to use the available hardware resources (VIOS, fabrics) for redundancy in the configuration. Both of these goals are accomplished by configuring a Storage Connectivity Group and Target Port Groups. 

Storage Connectivity Group

Create a Storage connectivity group in PowerVC with the following parameters (screenshots included below):

  • Details tab:
    • Boot Volume Connectivity Type = NPIV
    • Data Volume Connectivity Type = NPIV
    • Check box - Enable VM Deployment
  • VIOS members tab:
    • VIOS Redundancy for Volume connectivity = Exactly 2 VIOS
  • Fabrics tab:
    • NPIV Fabric Access Requirement = Every fabric per VIOS
    • Number of Fabric ports per VIOS = 1
    • Check box - Dynamically associate all fabrics that are connected to the VIOSes in this storage connectivity group.

Target Port Groups

The PowerVC user interface does not allow customizing a Storage Template for Pluggable storage drivers like the InfiniBox Cinder driver.

In order to define Target Port Groups for zoning associated with pluggable storage drivers, the PowerVC REST API must be used.

Step 1: Determine the Storage Template ID.

Create a powervcrc file with the correct environment variable definitions, like the example below. Replace OS_AUTH_URL, OS_USERNAME, OS_PASSWORD, and any other fields as appropriate for your PowerVC environment.

# Copy this file to your user's home directory and edit as necessary.
# In particular, you may wish to set these values:
#    - OS_USERNAME     : Your PowerVC user name
#    - OS_PASSWORD     : Your PowerVC password. If not set, openstack CLIs
#                        will prompt for the password as needed.
#    - OS_PROJECT_NAME : If you have multiple projects, specify which project
#                        you want to access; else you can leave this as
#                        ibm-default.
#
# NOTE: You should not add your password to this file until/unless its file
# permissions prevent other users from reading it. This is one reason to copy
# the file to your home directory and edit it there. Or you may wish to not
# add your password at all and have the CLI prompt you, as noted above.

export OS_IDENTITY_API_VERSION=3
export OS_AUTH_URL=https://powervc-lab.infinidat.com:5000/v3/
export OS_CACERT=/etc/pki/tls/certs/powervc.crt
export OS_REGION_NAME=RegionOne
export OS_PROJECT_DOMAIN_NAME=Default
export OS_PROJECT_NAME=ibm-default
export OS_TENANT_NAME=$OS_PROJECT_NAME
export OS_USER_DOMAIN_NAME=Default
export OS_USERNAME=root
export OS_PASSWORD=123456
export OS_COMPUTE_API_VERSION=2.46
export OS_NETWORK_API_VERSION=2.0
export OS_IMAGE_API_VERSION=2
export OS_VOLUME_API_VERSION=3
export PYTHONHTTPSVERIFY=0

As shown in the example below, source in this powervcrc file and run the cinder type-list command to show the storage template IDs:

# . powervcrc
# cinder type-list
+--------------------------------------+------------------------+-------------+-----------+
| ID                                   | Name                   | Description | Is_Public |
+--------------------------------------+------------------------+-------------+-----------+
| 0a0d337d-c964-400d-a8f8-df4e1599bc72 | ibox2233 base template | -           | True      |
| 0ebfc763-fbaa-4d4d-a1dc-6190eb92afaf | VNX666 base template   | -           | True      |
| 904dd9d8-4dc3-4bea-abda-4f209970538b | ibox2817 base template | -           | True      |
+--------------------------------------+------------------------+-------------+-----------+

Step 2: Determine the Project ID

Source in the powervcrc file to pick up the environment variables. Then run this python script:

#!/usr/bin/env python3

import requests
import json
import os
import urllib3
import sys
import cinderclient
urllib3.disable_warnings()
requests_timeout=10

# Create a session and authenticate
s = requests.Session()
headers = {"Content-Type": "application/json"}
url = os.environ["OS_AUTH_URL"] + '/auth/tokens?nocatalog'
project_name = os.environ["OS_PROJECT_NAME"]

payload = {"auth": {"identity": { "methods": ["password"],"password": {"user": {"domain": {"name": os.environ["OS_USER_DOMAIN_NAME"]},"name": os.environ["OS_USERNAME"], "password": os.environ["OS_PASSWORD"]} } }, "scope": { "project": { "domain": { "name": os.environ["OS_PROJECT_DOMAIN_NAME"]}, "name":  os.environ["OS_PROJECT_NAME"] } } } }
result = s.post(url, json=payload, headers=headers, verify=False, timeout=requests_timeout)
headers = {"Content-Type": "application/json", "X-Auth-Token": result.headers['X-Subject-Token']}

# Get the project id
url = "https://localhost:5000/v3/projects?name=%s" % project_name
print("GET %s" % url)
result = s.get(url, headers=headers, verify=False)
project_id = result.json()['projects'][0]['id']
print(project_id)

Step 3: Update the port groups for the InfiniBox storage template

As described in https://www.ibm.com/docs/en/powervc/2.0.1?topic=templates-configuring-storage-port-groups, PowerVC will use one of the defined port groups in a balanced fashion, and will rotate among them for each subsequent VM. The drivers:target_port_groups extra spec key defines the list of groups for the storage template, and the drivers:port_group:port_group_name key defines a port group. 

To update the port groups, use the cinder extra-specs-list command to view the current extra-specs, and then run the following python script to execute a REST API request to update the extra-specs.

The URL for the request is http://localhost:9000/v3/<project-id>/types/<storage-template-id>/extra_specs

# This is a continuation of the above python script. Save the above file as ibox_tpg.py and add these lines:
# Use storage template ID obtained earlier from cinder type-list command
storage_template_id = '904dd9d8-4dc3-4bea-abda-4f209970538b'
url = "http://localhost:9000/v3/%s/types/%s/extra_specs" % (project_id, storage_template_id)
payload = { 
    "extra_specs": 
    { 
        "drivers:target_port_groups": "PG01,PG02", 
        "drivers:port_group:PG01": "5742b0f0000b0111,5742b0f0000b0121,5742b0f0000b0131,5742b0f0000b0116,5742b0f0000b0126,5742b0f0000b0136",
        "drivers:port_group:PG02": "5742b0f0000b0112,5742b0f0000b0122,5742b0f0000b0132,5742b0f0000b0115,5742b0f0000b0125,5742b0f0000b0135"
    } 
}
print("POST %s\npayload: %s" % (url, json.dumps(payload, indent=4)))
result = s.post(url, json=payload, headers=headers, verify=False, timeout=requests_timeout)
print(json.dumps(result.json(), indent=4) if result.ok else result.status_code)

Step 4: Verify the port groups

Use the cinder extra-specs-list command to note that the extra-specs have been correctly updated.

Usage

Provision New Volumes from InfiniBox

Here is how to provision a new InfiniBox volume to an existing PowerVC VM:

  1. From Virtual Machines → VM list, select the VM and click on Attach volumes. 
  2. In the Attach volumes to VM dialog, Click on the New volumes tab.
  3. Type in a volume name, size (and count, if multiple volumes are needed. If count is supplied, the new volumes will have a suffix -1, -2, etc. attached to the name).
  4. From the Storage template drop-down list, select the InfiniBox base template
  5. Click "Attach Volumes"

Once the operation is successful, it results in the following modifications to the InfiniBox:

  1. a volume is created (or multiple volumes, if count was more than 1) 
  2. a host object is created and ports added to it (if this is the first time an InfiniBox volume is attached to this particular host).
  3. host is mapped to the volume

Migrate PowerVC volumes to InfiniBox

InfiniBox Cinder driver does not support the Cinder clone, migration, or volume retype APIs, so users cannot use PowerVC live capture or related PowerVC features. Instead, users must follow LVM-based migration or other migration processes that do not depend on those Cinder operations.

The process of migrating an existing PowerVC volume to InfiniBox is as follows:

  1. Create an appropriately sized Boot Volume or Data volume as described in the previous section
  2. Rescan so that the VM sees this new volume. 
  3. Use LVM migration to move the contents of the existing volume over to the InfiniBox volume. A good description of LVM migration procedure is provided in https://www.ibm.com/docs/en/ds8870/7.4?topic=manager-using-migratepv-command (make sure to select the appropriate OS version from the list on the top left).





Was this article helpful?
0 out of 0 found this helpful

0 out of 0 found this helpful

Last edited: 2021-08-19 21:33:33 UTC

Comments