Hi,
We have 2 Windows 2008 R2 servers running multiple IIS web sites and load balanced across Windows Network Load Balancer in unicast mode. Although there are two interfaces in each server, only 1 interface in each server participates in load balancing and other interface is used for a different backup LAN. The problem I am going to mention was not seen within the NLB for almost 1 year.
I have noticed intermittent "host unreachable" detected from NLB in each host from time to time since 3 weeks ago. After servers are rebooted, both hosts can be reached and can be detected from NLB manager. However it becomes unreachable in both servers within minutes and then becomes reachable again after several minutes. This behavior is noticed in the load balancer and pings do not work between the two hosts when the issue occurs. I did a packet capture to see what was going on with ARP message when the issue occurs. ARP entry goes missing in each server when the problem occurs and no ARP replies are returned from each server. But ARP requests are dispatched from both servers when the issue occurs. ARP replies come back after sometime after which hosts become reachable again.
I tried to create a permanent static ARP entry (By copying the MAC address from ARP table when the two hosts are reachable) in each host but that hasn't solved the issue either. It seems like the individual MAC address generated by each host is a virtual one and it doesn't seem to respond when the problem occurs.
However load balancing and web sites are fully functional without any issues even while "host unreachability" issue is detected.
Appreciate if someone could help me to dig the real problem out.
Thank you.