Aerospike Cluster goes down - FAILED ASSERTION (particle): (particle_map.c:857) map_asval_to_wire() sz -1 failed to serialize


We had a Cluster down scenario on our Production environment with the below error logs

Nov 11 2022 08:09:24 GMT: FAILED ASSERTION (particle): (particle_map.c:857) map_asval_to_wire() sz -1 failed to serialize

Nov 11 2022 08:09:24 GMT: WARNING (as): (signal.c:210) SIGUSR1 received, aborting Aerospike Community Edition build os ubuntu18.04

Our Aerospike Database is running as a service with a cluster of 2 on AWS ec2 AMD instance. There were no application deployments or instance changes during the same time frame. The CPU and RAM of the instance was also normal. Post the crash, the service never got back up and we had to manually restart the aerospike service.

BR Arvin

Are you leveraging UDFs by any chance?

I do see that there was an issue that was addressed in more recent versions of Aerospike (5.5+):

  • [AER-6519] - (UDF) A UDF that tries to return a list or map containing an as_rec element will cause an assertion.

This did cause an assertion that matches what you have observed. If you are not running UDFs, this may then be something else and I would first suggest upgrading to a more recent version as the version you are running (4.5) is out of maintenance window and there could be other issues that were fixed in more recent versions.