Hyper-V Failover Cluster in OpenStack

We’re very pleased to announce that starting with the Mitaka release we have introduced a new Openstack Nova Compute Driver, the Hyper-V Failover Cluster Driver, offering host based fault tolerance to tenants. The main advantage is that non high-available applications running in virtual machines can rely on the cluster for fault tolerance, a very common scenario in the enterprise IT world.

This feature is based on Windows Server Failover Clustering (WSFC) and is very useful when you need services that are always available, even if the Hyper-V compute hosts go offline due to planned maintenance or unplanned hardware or software issues. Thanks to this feature, the virtual machines are fault-tolerant and get automatically migrated to other nodes in the same cluster, where they continue to work unaffected.

All of this is now integrated with OpenStack, with the Hyper-V Compute Driver seamlessly notifying Nova about any failovers in the cluster, ensuring that instance status is properly up-to-date.

Windows Clustering is fully supported and automatically configured in our OpenStack Hyper-V hyper-converged product offer.

In the remaining part of this post, we’re showcasing the Failover Clustering feature in a quick demo that will work on any Mitaka OpenStack deployment. All you need is a minimum of two Hyper-V nodes (joined to an Active Directory domain) to add to your deployment.

Firstly, let’s boot some instances on the Hyper-V Cluster.

We can see the 3 instances with nova list:

And the same 3 instances in the Failover Cluster Manager:

Now, we’re going to forcefully restart the HV12R202 Hyper-V host server where two of the three instances are hosted. Being a planned restart, the instances hosted on this node will live migrate to other Hyper-V nodes in the cluster without any downtime. This is the so-called “node drain” feature. If instead the Hyper-V server undergoes an unplanned shutdown, the instances will failover to other nodes.

The instances have been live migrated to the other nodes in the cluster as we can see in the Cluster Manager shown in the following snapshot:

A snippet from the Hyper-V Nova Compute logs, showcasing the occurring failover events.

The changes are consistent with Nova and the instances location has been updated.

Of course, you can try out this feature yourself. We’re going to explain the setup in the following part.

Requirements

• OpenStack Mitaka deployment
• Active Directory
• At least 2 Hyper-V compute nodes in the same Active Directory, running Windows Hyper-V / Server 2012 R2 or newer, including Nano Server
• Install the OpenStack Hyper-V Compute Driver on the Hyper-V nodes
• Optionally, you can add Scale-Out File Server storage nodes, also joined to the Active Directory domain and part of the same cluster.

Setup

1. Run the following PowerShell script on a server in the Active Directory domain. This script will configure the Failover Clustering service on each Hyper-V compute node along with a SMB share, where the instances’ disks and configurations will be hosted on, since they will have to be accessible from each node. The shared storage used by Hyper-V can be either based on SMB or CSV, for this basic demo we’ll just use a simple SMB share, take a look at our OpenStack Hyper-V hyper-converged offer for a full Scale-Out File Server storage configuration.
   
   ######################## # Windows Server 2012 R2 # setting up server for share # please setup your own configurations here. $hyperv_nodes = @("HYPERV_NODE_01.DOMAIN", "HYPERV_NODE_02.DOMAIN") $cluster_name = "CLUSTER_NAME" $cluster_user = "DOMAIN\USER" $smb_path = "S:\SMB_SHARE" $smb_name = "SMB_SHARE" # install locally Install-WindowsFeature File-Services, FS-FileServer # this will install the features necessary, including the Failover Clustering feature. Invoke-Command -ComputerName $hyperv_nodes {Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering} New-Cluster –Name $cluster_name -Node $hyperv_nodes # Create share folder MD "$smb_path" # Create file share New-SmbShare -Name $smb_name -Path $smb_path -FullAccess $cluster_user # Set NTFS permissions from the file share permissions Set-SmbPathAcl –ShareName $smb_name

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

   ######################## # Windows Server 2012 R2 # setting up server for share   # please setup your own configurations here. $hyperv_nodes = @("HYPERV_NODE_01.DOMAIN", "HYPERV_NODE_02.DOMAIN") $cluster_name = "CLUSTER_NAME" $cluster_user = "DOMAIN\USER" $smb_path = "S:\SMB_SHARE" $smb_name = "SMB_SHARE"   # install locally Install-WindowsFeature File-Services, FS-FileServer   # this will install the features necessary, including the Failover Clustering feature. Invoke-Command -ComputerName $hyperv_nodes {Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering}   New-Cluster –Name $cluster_name -Node $hyperv_nodes   # Create share folder MD "$smb_path" # Create file share New-SmbShare -Name $smb_name -Path $smb_path -FullAccess $cluster_user # Set NTFS permissions from the file share permissions Set-SmbPathAcl –ShareName $smb_name

2. On each Hyper-V compute node, we must configure the nova-compute service to use the Hyper-V Cluster Driver along with a shared instances configuration path. Update these configuration options with your own values and add them to their appropriate sections:
   
   # add extra configs in nova.conf [DEFAULT] compute_driver=hyperv.nova.cluster.driver.HyperVClusterDriver instances_path=\\SHARE_HOST_IP\SHARE_PATH\Instances

   1 2 3 4

   # add extra configs in nova.conf [DEFAULT] compute_driver=hyperv.nova.cluster.driver.HyperVClusterDriver instances_path=\\SHARE_HOST_IP\SHARE_PATH\Instances

3. (Optional) Install the OpenStack Mitaka Cinder Storage driver, enabling SMB3 block storage volumes on clustered storage with Windows Server Scale-Out File Server enabled.
4. Done! Now you can create and use your OpenStack Hyper-V instances as usual, with the additional peace of mind provided by the Hyper-V Clustering feature working behind the scenes.

We hope you enjoy this feature and if you have any questions or suggestions, please let us know!