Virtual IP in clustered environment stops responding

Problem Description:

All of a sudden you may stop getting the ping response from virtual IP used in the clustered environment. It generally happens when node owning the IP resource transfers the virtual IP to its partner because of some reason or network glitch.

Solution:

As the virtual IP and the mac is always cached in network devices and when virtual IP is transferred to another node (even if you transfer it manually) you will not get response from virtual IP from another VLAN/Subnet, however, you can access (or PING) the virtual IP from same VLAN/subnet.

If you have the cluster of the physical server then restart the nodes one by one, this will help clear the cached mac and IP table from network devices, if your servers are super quick in the boot process the shutdown the node for 5 minutes and then start.

If you are working on a cluster having virtual machines then restarting the VMs will not fix the issue, VMs are very quick in the boot process and then starts sending their mac immediately in the network, since it doesn’t break the connection to the network device, the IP to mac cache never gets cleared.  Shutdown the virtual machine (node) in cluster one by one. Start the virtual machine after five minutes it will give enough time for VM to break its mac advertisement in the network, clearing the mac to IP cache.

When your machine boots up, go to cluster management and bring the IP resource online, you can access the IP from network now.