For several months I have had a Server 2012 R2 Failover Cluster which has been successfully live migrating virtual machines between the nodes. This past week I've noticed they are no longer migrating successfully. Unfortunately, the event logs are not being very helpful.
On the Destination node I get two entries in the event log:
Source: Microsoft-Windows-FailoverClustering
Event ID: 1069
Cluster resource 'Virtual Machine XYZ' of type 'Virtual Machine' in clustered role 'XYZ' failed.
Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it. Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet.
Source: Microsoft-Windows-FailoverClustering
Event ID: 1205
The Cluster service failed to bring clustered role 'XYZ' completely online or offline. One or more resources may be in a failed state. This may impact the availability of the clustered role.
On the Source node, I get one event log entry:
Source: Microsoft-Windows-Hyper-V-High-Availability
Event ID: 21502
Live migration of 'Virtual Machine XYZ' failed.
Virtual machine migration operation for 'XYZ' failed at migration destination 'Node-A'. (Virtual machine ID 2B038DBB-6BC7-4225-B5F3-FBC8324E302C)
I am not seeing any other entries in the System Event Log occurring at the same time on either the source or destination nodes. The Cluster Validation Wizard confirms the cluster is configured correctly and functioning, although I did skip the CSV tests as I'll need to schedule a maintenance time before knocking the VMs offline.
The cluster consists of 4 nodes all running Server 2012 R2 Datacenter edition. All of the virtual machines are configured to allow migration to processors of different versions as 2 nodes are running a newer generation of Intel processors than the 2 older nodes. All of the VMs are configured to use Dynamic Memory. Both CSVs (the VM storage volume and the Disk Witness volume) are connected via iSCSI.