I've recently upgraded my 2008R2 cluster to 2012R2, which has been one problem after another. I'm at the final stages trying to add my last node into the cluster and it's failing. Right now I was able to add 2 nodes in no problem, but this final node is giving me problems. The whole cluster is built with fresh installs of Hyper-V Server 2012R2. The 2 active nodes are working fine, live migrating as well. When I try to add this node it hangs at the "waiting for confirmation that this node is a fully functioning member of the cluster" until it times out with an error code of 0x5b4. I'll also see the following warnings in the FailoverCluster Diagnostic log on the problem node:
"Event ID 2050: cxl::ConnectWorker::operator (): (1460)' because of '[FTI][Follower] Aborting connection because NetFT route to node PAVER on virtual IP fe80::cd75:bc4d:bf38:e97c:~3343~ has failed to come up.'"
"cxl::ConnectWorker::operator (): (1460)' because of '[FTI][Follower] Aborting connection because NetFT route to node RECLAIMER on virtual IP fe80::ccf1:302:b63b:3f02:~3343~ has failed to come up.'"
Paver and Reclaimer are the other 2 working nodes in the cluster. Subgrade is the 3rd, failing one.
I'll also see this error: Event Id 2051 [QUORUM] An attempt to form cluster failed due to insufficient quorum votes. Try starting additional cluster node(s) with current vote or as a last resort use Force Quorum option to start the cluster. Look below for quorum information,
In the main system log I'll see: Event ID 1653 "Cluster node SUBGRADE could not to join the cluster because it failed to communicate over the network with any other node in the cluster. Verify the network connectivity and configuration of any network firewalls."
It seems to be network related but all servers can ping, I've disabled the firewall on all servers, all nodes have the same virtual switch config, DNS is functioning. There is a quorum disk and the cluster passes all validation tests when I attempt
to add this 3rd node. So I'm at a loss. Any ideas? Could this be some IPv6 issue? since according to the logs it's trying to use IPv6 to communicate during setup? I don't use IPv6 for anything myself. I'm hesitant to disable
that on all nodes for fear I'll break the 2 functioning nodes. And if it is an IPv6 problem it would seem odd that the first 2 nodes would add just fine.
Thanks,
Todd