I have a test Aerospike cluster with one node and several namespaces. I am testing a new configuration change, to increase the high-water-memory-pct from 70 to 90%.
When I apply this change and restart the Aerospike server, I notice that the high-water-memory-pct reported by the Datadog metrics agent is 75.
I tested by setting a lower value: 72%. This change is reflected in the metrics.
This does not happen in other clusters I have tested on.
Why is this happening? Is there a global setting that is overriding the settings I have provided?
So it is set to 75% but you expected it to be 90% per your config file? I tried replicating this but was unable to. I’m wondering if your config file has the setting in the wrong context. Could you share your config?
Or share the command used to change the config and the output of the info command reporting on the config value:
asinfo -v "get-config:context=namespace;id=test" -l (replace test with your namespace id).
Or directly from asadm:
show stat config like high-water
I am wondering whether this could be something related to the datadog agent itself, but as you indicated you are running the Enterprise Edition, I would encourage you to reach out to support to get to the bottom of this. There shouldn’t be anything that overwrites or limits the eviction related thresholds (the …-high-water-… config params).
They’re saying they apply the config change and restart. I wonder if they’re setting it dynamically and then restarting, which then loads the value from config…
so dynamic set to 90, then restart and reload config which has a value of 75? its not 100% clear to me though