Aerospike not forming cluster after upgrading to version 3.4

Operations
1

Hi,

We have two nodes which used to work together as a cluster in multicast mode. We have upgraded them to version 3.4. After installation the Nodes could not form a cluster.

I have also tried mesh setup, also without success. Can you help?

2

hi haniq,

Was there any cluster configuration change done? Do both the nodes have the same namespace and network configuration? Any other change other than the upgrade? Any network changes?

Also, is there any warning in the logs?

3

Hi Anshu,

Problem solved. True, there was a slight difference in namespaces which seemed not to cause an issue on the older version. I matched them and its fine now.

Thanks.

1 Like
4

hi Haniq,

Goo to know that the issue is resolved. However please note that a namespace mismatch will always cause problems in cluster formation. What would have happened in the older version is that the older server was not restarted after the config file change, and hence the config with changed namespace never came into effect. The restart post upgrade caused the config reload and hence the failure to join cluster.

5

Just to check… If you have a two-node cluster, and for some reason you want to add or remove a namespace from the configurations. Can it be done gradually without bringing the whole cluster offline.

As in the scenario we had, one of the servers had an extra namespace, the cluster failed to form. How to add a namespace to the two of them without a cluster timeout?

6

Hi Haniq,

  1. We will need to bring down the cluster completely since the nodes with namespace mismatch will not be able to join the cluster. Thus the reason for going to a planning phase before going in production.

  2. Dynamically adding namespace is not currently possible since a cluster restart is required for the new configuration to be accepted.

This should provide more information.