8 nodes and with disk storage. with 5000~20000 reads tps and 20000 ~ 100000 udf tps.
always crash with logs above. any one help me will be appreciated.
Running at CentOS 7.9.2009 and Linux 3.10.0-1062.4.3.e17.x86_64
The logs leading up to the crash would help. Has this always been an issue? Did you recently upgrade or start utilizing some feature? Are hardware logs clean? Is it only 1 node? It looks like 5.7.0.18 is the latest, so there have been a number of bug fixes since then which may be relevant Aerospike Server CE Release Note | Download | Aerospike
1 Like
Thanks for reply.
I have checked the update logs after 5.7.0.10 but can not find any info about this question.
It happens on most of (always 4~6) my 8 nodes, and when with continous high(40000 ~ 80000) tps. I got a core dump file and trace it with gdb, I found it try to get the particle size of a bin with particle type show as 0,
pic link: gdp info
and will cause NULL Pointer at flow code:
as code
because the 0 response as AS_PARTICLE_TYPE_NULL in particle_vtable is always point to NULL
but
I check the code at tag 5.7.0.18 and branch master, the code is the same. In my case I always use bin in udf as a map create by aerospike udf api, I wonder why the particle->type is AS_PARTICLE_TYPE_NULL and how cause it. Any help will be appreciated.
Would you be able to provide code to reproduce this crash?
1 Like
Hi, could we have that core dump and the asd binary that you were using? Thank you.
1 Like
Thanks your reply The file is on the production server. I’ll try to apply to get the file, it will take some time. Could I have your email so that I can send it to you privately after I get it.
That would be most helpful, I’ve sent you a message with my information.
1 Like
I’ve send an email with the core file.
Thank you, we have found the bug thanks to your core file. We’ll have it hot-fixed.
2 Likes
I’m glad it helped you and thanks for your attention on my question. 
How do I know in which version this issus is fixed?
It was addressed in version 5.7.0.19:
- [AER-6555] - (UDF) For namespaces with data not in memory, a UDF that writes multiple bins and would exceed
write-block-sizemay cause a crash.
It was also hotfixed in versions 6.0.0.3 and 5.6.0.23.
1 Like