We have recently configured a two-node Failover Cluster for a client (a large multi-campus university). It is a Storage Spaces Direct cluster and it runs HyperV virtual machines. It is Windows Server 2016.
The two servers sit in the same rack and are connected through two switches, which also sit in that rack. We were hoping that with this configuration the virtual machines would have fairly decent availability.
Yesterday the client had a mishap during an attempt to update firmware in a router. This router connects the cluster with the rest of their infrastructure, including:
- DNS Servers
- Domain Controllers
- The file server, which we use as a file share witness in the cluster.
The result of losing that connection for 5-10 minutes was that all the virtual machines in the cluster stopped abruptly (no proper shutdown). They were automatically started again once the network connection was re-established, but obviously it was not a nice experience.
A few questions:
Is this expected behavior?
To what degree are failover clusters dependent on access to domain controllers, DNS Servers and witnesses (in the two-node configuration) for their continued operations?
Could the stoppage of all virtual machines have been avoided if the file server that acts as the cluster witness was sitting inside the same rack and was directly connected to the same two switches as the cluster servers? I am thinking that it would not help because is was added as a witness with its share-name, so it is likely dependent on DNS lookup to access it.
Would it even help to add more nodes to the cluster? I realise that many frown upon the 2-node setup, but I suspect that having more nodes would not help in this case.
Are there recommendations for how to do something like this? Should we add a domain controller as a virtual machine in the cluster? Would that have avoided the vm stoppage? The folks that manage the AD at the client are apparently very restrictive about locations of domain controllers, so this is not something that we can easily do.