My cluster is having problems at Amazon EC2


My cluster is showing migrations, but my nodes appear to be up. What is happening?


Technically speaking migrations occur when the cluster state has changed. Sometimes this can happen when a heartbeat has not been heard from a server for a while, but then the server rejoins. So the cluster may appear like it is up and whole.

This situation is most common when there are networking issues. At Amazon, this could be common. The best way around this is to make the number of missed heartbeats larger. In the configuration file /etc/citrusleaf/citrusleaf.conf, simply make sure that the “internal” and “timeout” are at the values below.

network {
   heartbeat {
      interval 150
      timeout 50