Uneven memory usage distribution

in which circumstances , the partition would store more keys but used less memory . we had a cluster with three nodes , but there is a huge memory usage difference

how does this happen and how to avoid it (community version ,not enterprise)

Do you have secondary indices defined by any chance? Could it be a case of memory not garbage collected on the secondary indices and the node which is using much less memory may have been added after the other ones or recently restarted?

the node which use less memory is added recently , but after that , we add another two new nodes yestoday, and the new two nodes looks good

and we do not use secondery index , is there any other possibilities for this kind of issue ? thank you very much

So, next step would be to confirm where the memory is used. I assume your namespace is configured with data-in-memory true, right? We can first confirm how the memory is distributed, to make sure nothing unexpected.

What does the following command show (to be run from asadm)

Admin> show stat like memory_used
~test Namespace Statistics (2020-08-25 04:06:38 UTC)~
NODE                    :   d7af462a411d:3000
memory_used_bytes       :  0
memory_used_data_bytes  :   0
memory_used_index_bytes :  0
memory_used_sindex_bytes:  0

But actually since the disk usage seems similar, could it be that this one node is simply configured with data-in-memory false and the other ones are configured with data-in-memory true?

1 Like

I think this last theory should be the right one. I hope so… since ~14 GiB on disk would likely be ~8-9 GiB in memory which is the gap on that node that is using less memory.

1 Like

you are awsome , thank you very much . it does misconfigured cause that issue , I lost the data-in-memory setting.

these issue borther me a long time ,thank you very very much for your kindness and patience

All the best wishes for you : )

This topic was automatically closed 6 days after the last reply. New replies are no longer allowed.

© 2015 Copyright Aerospike, Inc. | All rights reserved. Creators of the Aerospike Database.