Hello! we’re running Aerospike Community 5.5.0 with a 4 node cluster
One day we suddenly started seeing an increase in CPU usage in only one of the nodes
For some reason the leader suddenly has higher cpu usage:
Anyone had a similar experience? How do I debug this?There doesn’t seem to be any error messages or anything
Could be any number of things going on. Are they all running on the same exact CPU type? Is the QPS and workload even across all nodes, or could there be hotspots? Are they all the same server version and config? You could try to compare perf top -p $(pidof asd) --call-graph dwarf -F 33 to compare CPU profiles between nodes, but I’m not sure how that’ll work in K8s. What troubleshooting have you done? If you drain, quiesce, or replace aerospike-3 does that hotspot move and then move back?
Hey! Thanks for the reply It was the exact same setup, each one has a node of it’s own with the exact same specs I can no longer replicate this, I drained the node and when it went on the new one the cpu was even. My guess it’s just a noisy neighbor on GCP’s hardware or something. Anyways thanks for the help
