Aerospike Crash



Today one of our cluster member crashed. Were running two nodes, one of it went offline.

Were running on the latest version ( enterprise).

I attached you all logfiles / configs we got. Please investigate.


We will investigate on the logs and the stack traces. From a quick look, it seems that the cluster was showing Cluster integrity faults about 4 minutes prior to the crash and the node that crashed was running as a 1 node cluster. Was there anything unexpected triggered on the other node?

Starting, we have made improvements in the paxos algorithm implementation and we recommend the configuration paxos-recovery-policy to 'auto-reset-master' if cluster is sensitive to network blips.

In order to investigate further, could you please share information on the features you are currently using - UDF, Scans, Batch operations and if anything changed close to when the node crashed?

Are you currently deployed on AWS/similar or bare-metal?


We identified a fix for the SegV that you observed that missed the release that you used. It has gone out in release 3.7.1. Please give it a spin and let us know if you run into issues.

[AER-4487], [AER-4690] - (Clustering/Migration) Race condition causing incorrect heartbeat fd saved and later not removable.


Can you indicate what type of access pattern you have? purely put/gets? batches? scan? secondary index?


It was unexpectedly starting to swap its memory (not caused by Aerospike).

Bare metal.

~70% read, 30% write. Only some batch gets and scans per hour. No secondary index.

Will do!