What exactly triggers automatic node adding?


#1

So I had created a first node on a virtual machine (bridge connection, own IP)… when I got it right, I created another node on a dedicated machine, and both were working fine independently…

And then, a few days after, this happened : AMC Screenshot

Now, I’m not that surprised, I remember reading somewhere that Aerospike was rack-aware, and added new nodes to the cluster as they were plugged in. What I’m wondering is, why did it take so long for AS to do it? How does it work under the hood exactly?

Thanks in advance!


Adding new node in mesh mode
#2

Each node discovers other nodes through the heartbeat protocol. By default this is configured to multicast and I assume that it still is in your case since you were not expecting the cluster to form. If it were configured to mesh then at least one of your nodes would need to be configured to know about the other by using the mesh-seed-address-port parameter.


#3

Thanks.

Something new has happened (it’s funny how many things happen just by letting it run…)… Both nodes visibility were false. I checked it out and it seemed to be because both nodes were being accessed and that it was messing up with the configuration. So I set the access-address to 19.168.1.20 for both nodes, and now the AMC show this : AMC Screeshot.

And when I check it out, indeed: Console Screenshot.

I’m not really good at interpreting mixed signals, so I’m not sure about what to do here…

Thanks in advance.

Edit: I managed to get both nodes running again by going back to my previous configuration, but when I try to “merge” the nodes into a single cluster, I get this message: AMC Screenshot.


#4

Cluster visibility False indicates that the nodes that AMC can see are not exactly the same as the nodes the Aerospike nodes see. This isn’t necessarily critical, in your case I suspect it is because your nodes have multiple NICs (?) and Aerospike is reporting all interfaces to the client. The client AMC is using would then interpret the extra interfaces as extra nodes that it isn’t aware of.

Would need to see your Aerospike log output to determine what happened here.

Would seem these nodes haven’t discovered each other yet. There is a chance the logs may be able to provide a hint of why the cluster isn’t forming, however; at the very least, I would be able to determine if the nodes have heard from one another by inspecting your logs.