InfiniBox system features a virtualized storage service that is shared by all hosts and applications. InfiniBox accommodates high-performance requirements for multiple applications with varying performance objectives allowing for fair resource allocation. QoS is available either at the single volume level, or all of the volumes within a single pool. The QoS feature aligns the system performance to the varying business needs, by enabling the user to restrict IOPS and throughput, or both to the relevant entity. In addition, QoS solves the noisy neighbor problem by limiting the resources the entity can use. The Quality of Service (QoS) provides a facility to limit the performance of volumes (either individually or as part of a pool) on a per IOPS and/or throughput basis.
The QoS feature allows to establish performance tiers and to sustain high-performance needs, allowing the user to differentiate performance levels to their applications.
The QoS policy
The QoS policy is a logical entity that sets performance limits levels that can later be assigned to datasets. The QoS policy levels are comprised of the following:
- Max IOPS - the maximum number of IOPS that are allowed for a volume
- Max throughput - the maximum throughput that is allowed for a volume
- Burst factor - the factor of increased performance during the burst. The burst cannot exceed this factor. See also: Burst
Any volume, or pool, in the InfiniBox system can be assigned to, or removed from a QoS policy.
The QoS policy applies restrictions for both FC-originated IOPS and throughput and iSCSI.
QoS configuration limits
See: QoS limits.
What happens when the limit is reached
In case that the actual performance exceeds the performance limit (as the host keeps sending requests), InfiniBox takes measures to slow down the pace of the requests. InfiniBox applies artificial latency that moderately returns the pace of requests to the amount that is allowed according to the policy. The added latency is proportional to the amount of IOPS or throughput that the entity tries to consume above the limit. The effect of the added latency on the system resources is minimal.
If the host still does not slow down the pace of the requests, InfiniBox will reject the extraneous IO requests (using the DEVICE_BUSY response).
The Burst option allows the entity with an assigned QoS policy to consume more IOPS (or throughput) than the Max IOPS (and Max Throughput) for a short period of time. An entity that has not consumed the entire allowed performance gains allowance that can be used later to exceed the QoS max limits. The burst factor determines the amount of performance that the entity (either a volume or a pool) is allowed to consume above its max limits. The actual IOPS (or Throughput) during the burst cannot exceed the multiplication of the burst factor by the max IOPS (or Throughput).
The burst allowance is determined by the lowest of the following two factors:
- Amount of non-consumed IOPS (or throughput)
- The burst factor that is set in the policy
Amount of non-consumed IOPS (or throughput)
The following picture shows the IOPS consumed by an entity along a specified period of time. On the graph, two time frames are marked, denoting the following:
- The area marked with 1, represents the number of IOPS (or throughput) that the entity did not consume although the policy allowed that
- The area marked with 2, represents the number of IOPS (or throughput) that is available for the entity, for above-the-maximum consumption. This number is proportional to the amount of not-consumed IOPS (or throughput)
The burst is also limited by the burst factor:
QoS policy and special SCSI commands
Some SCSI commands are not limited by the QoS policy due to their special nature, or the way they are utilized.
SCSI commands that are not accounted for credits and are not limited:
SCSI commands that are not accounted for throughput (they are accounted for IOPS only):
- XCOPY - accounted only for the source volume IOPS
QoS policy and replicated volumes
The IOPS (and throughput) that the source volume receives from the host are accounted for the QoS. The replication-associated operations (reads from the source and writes to the target) that occur during the replication are not accounted for. This is the case for both asynchronous and synchronous replication.
Note: The replication throughput rate can be limited from the Network Space. To set the network space rate limit, see: Modifying a network space.
Assigning a QoS policy to a pool
When the pool is assigned to a policy, all of its volumes cumulatively share the same policy limits. The performance limits imposed by the pool QoS policy apply to all of the pool's volumes. IOPS (and throughput) that are consumed by a specific volume, are deducted from the IOPS (and throughput) that are available to all other volumes of the pool. When the pool reaches the limit of IOPS (or throughput), all of the pool volumes are affected. IOPS (or throughput) to all of the pool volumes is slowed-down (or even rejected, as explained above), regardless of their previous IOPS (or throughput) consumption (first-come, first-served).
In the following example, a pool with a QoS policy with max IOPS has 3 volumes. Each of the 3 volumes accepts 1,000 IOPS. The graph below displays a stacked view of the volumes IOPS, where cumulatively the 3 volumes accept 3,000 IOPS. At this point (time t0)The cumulative IOPS of all of the volumes is below the policy's max IOPS limit of 5,000. Next, at time t1, each of the volumes accepts an increasing amount of IOPS. If the volumes did not have the constraint of the pool QoS policy, they could accept more IOPS. However, as the volumes are subject to the pool QoS policy, they fairly share the allowed IOPS (5,000 IOPS) between them.
If the pool-volumes QoS policy allows burst, the burst is allowed to all of the pool's volumes.
The volume can be policed by a single volume policy and a single pool-volumes policy (that is imposed on the pool that the volume belongs to) at the same time. A volume that is both individually policed and belongs to a policed pool, is affected by both policies. Whenever one of the limits is reached, the volume performance is affected.
Note: only the pool's volumes are affected by the pool's QoS policy. The pool's filesystems are not affected. Their OPS and throughput are not accounted.