Q: How can I perform a shared nothing live migration from one cluster to another cluster?

A: Shared nothing live migration allows the migration of a virtual machine (VM), including its storage, memory, and device state, between Hyper-V hosts without any downtime. A logical extension is the desire to perform a shared nothing live migration between Hyper-V clusters (when you don't want to use a cluster migration). However, you can't use shared nothing live migration to migrate between clusters directly. Instead, you need to do the following:

  1. Remove the VM as a cluster resource.
  2. Use shared nothing live migration to move the VM to a node on the new cluster on cluster storage.
  3. Add the VM as a cluster resource on the target cluster.

There's no downtime for any of these actions, because VMs can be added and removed from clusters without incurring downtime. I wrote a script that uses a basic flow to perform this action, as follows:

  1. Prompts you for a credential that it will use on the remote cluster
  2. Prompts for a node of the remote cluster you want to migrate the VMs to
  3. Enables CredSSP delegation to the node specified (you need to have enabled CredSSP on the remote server, using the Enable-WSManCredSSP Server command, before starting this)
  4. Starts a loop that prompts for a VM name to be moved and then:
  • Finds the cluster group that owns the VM and displays the cluster resources for the VM
  • If the VM isn't running on the current node, the VM is live migrated to the current node (which makes things simple)
  • The VM is removed from the cluster
  • There's a step that can be commented out, which stops the VM—this is necessary if the processors in the source and target cluster don't match (e.g., Intel versus AMD or different versions of Intel/AMD and you don't have processor compatibility enabled)
  • The VM is moved to the target node using shared nothing live migration
  • The VM is made a cluster resource on the target cluster
  • The next VM name is prompted on, or type Q to quit, and the earlier steps then repeat for the next VM

The complete script follows. Note that you need to run the script on a node in the source cluster. In addition, you might want to change the storage location that's used to store the VM on the target from the C:\ClusterStorage\Volume1 target that I hard-coded into the script. Also, make sure the Hyper-V Module for Windows PowerShell and the Failover Cluster Module for Windows PowerShell are installed on the host running the script. This script performs no error checking; it's a very basic script, so if you intend to use it in a major way you should probably add some error checking. My goal here is simply to show the required steps.

  1. #Migrates virtual machines between clusters
  2. #Copyright 2014 John Savill
  3. #
  4. #Assumes virtual switch with same name exists on target
  5. #
  6. #Need to enable CredSSP to solve the double-hop problem associated with cluster commands
  7. #http://blogs.msdn.com/b/clustering/archive/2009/06/25/9803001.aspx documents the double-hop problem related to clustering remote commands
  8. #On the target server must have run Enable-WSManCredSSP Server
  9.  
  10. #Lets get started
  11.  
  12. Write-Host 'Please enter credential to be used for remote cluster execution'
  13. $CustomCred = Get-Credential #Get credentials to use with CredSSP on the remote cluster
  14.  
  15. Write-Host 'Ensure the target Server has been configured to accept CredSSP by running command Enable-ESManCredSSP Server on the server'
  16.  
  17. $DestinationHost = Read-Host 'Enter the target host that will be used for all migrations >'
  18.  
  19. Enable-WSManCredSSP Client -DelegateComputer $DestinationHost -Force
  20.  
  21. $VMName = Read-Host 'Enter VM name to move >'
  22. while ($VMName -ne 'Q')
  23. {
  24.  
  25.     #Show the cluster resources. Cannot use the $VMName as the cluster group as may not match, for example SCVMM names the cluster group differently
  26.     #Get-ClusterGroup -Name $VMName | Get-ClusterResource
  27.     $ClusterGroups = Get-ClusterGroup | ? {($_.GroupType -eq 'VirtualMachine') -and ($_.State -eq 'Online')}
  28.     foreach ($ClusterGroup in $ClusterGroups)
  29.     {
  30.         $TempVM = $ClusterGroup | Get-VM
  31.         if ($TempVM.Name.ToUpper() -eq $VMName.ToUpper())
  32.         {
  33.             $FinalClusterGroup = $ClusterGroup
  34.         }
  35.     }
  36.  
  37.     $FinalClusterGroup | Get-ClusterResource
  38.  
  39.     $ClusterGroupName = $FinalClusterGroup.Name
  40.  
  41.     #Store the host the VM is currently active on if wanted to take the approach of NOT moving it to local node but means requires Constrained Delegation configured
  42.     $NodeName = Get-ClusterGroup -Name $ClusterGroupName | Get-ClusterResource | Where-Object {$_.ResourceType -eq "Virtual Machine"} | Select-Object OwnerNode
  43.     $CurrentNodeName = $NodeName.OwnerNode.Name
  44.  
  45.     #View the VM
  46.     Get-VM -Name $VMName -ComputerName $CurrentNodeName
  47.  
  48.  
  49.     #Move the VM to the current host
  50.     $LocalNodeName = $env:COMPUTERNAME
  51.     if ($CurrentNodeName.ToUpper() -ne $LocalNodeName.ToUpper())
  52.     {
  53.         Write-Host 'Moving VM from '$CurrentNodeName' to current node'
  54.         Move-ClusterVirtualMachineRole -Name $ClusterGroupName -Node $LocalNodeName -MigrationType Live
  55.     }
  56.  
  57.     #Remove the cluster resources and group which does not stop or remove the actual VM on the host
  58.     Write-Host 'Removing cluster group for VM'
  59.     Remove-ClusterGroup -Name $ClusterGroupName -RemoveResources -Force
  60.  
  61.     #Sleep to avoid lock from removing cluster resources (not typically required)
  62.     Start-Sleep -Seconds 10
  63.  
  64.     #Have to shutdown as different processor type (Intel to AMD in my case but normally would not need to stop the VM) or if different versions
  65.     #and processor compatibility not enabled
  66.     Stop-Vm -Name $VMName #This will wait until the VM has stopped
  67.     Start-Sleep -Seconds 10
  68.  
  69.     #Now migrate it from this node so don't need -ComputerName $LocalNodeName
  70.     Write-Host 'Moving VM to target node'
  71.     Move-VM -Name $VMName -DestinationHost $DestinationHost -DestinationStoragePath C:\ClusterStorage\Volume1\$VMName
  72.  
  73.     $ScriptBlockContent = {
  74.         param ($VMName)
  75.         Add-ClusterVirtualMachineRole -VirtualMachine $VMName -Name $VMName }
  76.  
  77.     #Now add as resource on the target cluster
  78.     Write-Host 'Making VM a cluster object on target'
  79.     Invoke-Command -ComputerName $DestinationHost -ScriptBlock $ScriptBlockContent -ArgumentList $VMName -Authentication Credssp -Credential $CustomCred
  80.  
  81.     #Could automatically start the VM here if you wanted but I don't in this example
  82.  
  83.     #Get next name
  84.     $VMName = Read-Host 'Enter VM name to move (Q to quit)>'
  85.     $VMName = $VMName.ToString()
  86. }

The following is an example execution moving a single VM.

  1. PS C:\> .\MigrateBetweenClusters.ps1
  2.  
  3.  
  4. cfg         : http://schemas.microsoft.com/wbem/wsman/1/config/client/auth
  5. lang        : en-US
  6. Basic       : true
  7. Digest      : true
  8. Kerberos    : true
  9. Negotiate   : true
  10. Certificate : true
  11. CredSSP     : true
  12.  
  13. Please enter credential to be used for remote cluster execution
  14. Ensure the target Server has been configured to accept CredSSP by running command Enable-ESManCredSSP Server on the server
  15. Enter the target host that will be used for all migrations >: savaushv10
  16. Enter VM name to move >: Test - DC01
  17. Name         : SCVMM DC01 (1)
  18. State        : Online
  19. OwnerGroup   : SCVMM DC01 Resources (1)
  20. ResourceType : Virtual Machine
  21.  
  22. Name         : SCVMM DC01 Configuration (1)
  23. State        : Online
  24. OwnerGroup   : SCVMM DC01 Resources (1)
  25. ResourceType : Virtual Machine Configuration
  26.  
  27. Name             : Test - DC01
  28. State            : Running
  29. CpuUsage         : 0
  30. MemoryAssigned   : 883949568
  31. MemoryDemand     : 724566016
  32. MemoryStatus     : OK
  33. Uptime           : 81.20:38:09
  34. Status           : Operating normally
  35. ReplicationState : Disabled
  36.  
  37. Moving VM from  savdalhv11 to current node
  38. Name      : SCVMM DC01 Resources (1)
  39. OwnerNode : savdalhv10
  40. State     : Online
  41.  
  42. Removing cluster group for VM
  43. Moving VM to target node
  44. Making VM a cluster object on target
  45. Name           : Test - DC01
  46. OwnerNode      : savaushv10
  47. State          : Offline
  48. PSComputerName : savaushv10
  49.  
  50. Enter VM name to move (Q to quit)>: Q