Hi team,
We have almost 7-8 nodes of Aerospike Community edition 4.0.0.5 and we have almost 2bilion objects on every node. The thing is, as I shared the screenshot of the AMC, in 3 nodes, the data is not balanced.
All the nodes are the same as configs, resources. Please let me know if you need any additional information or configuration to paste here.
Screenshot: AEROSPIKE — ImgBB
prefer-uniform-balance in 4.3.0.2 might help, but that requires enterprise. There’s not a lot to go on here just off a picture… but if I had to guess, based off the distribution, you probably had lost data that was specifically homed in that node more than others. This can occur if you, for example, wipe out an RF=1 namespace on a single node but not the rest (and dont allow migrations to occur) or really if you’re just wiping data without restarting in general this can occur… Was there any data-loss event recently? When did it occur? Do you have logs from when it went from balanced to unbalanced? Do you use rack-aware (guessing not, but it could be a cause)?